Thursday, November 4, 2010

Neo4j Internals: Transactions (Part 3) as a complete run and a conclusion

This is the last post in the series, at least the core one. Here I will try to follow a path from the initialization of the db engine and through the begin() of a transaction and creation of a Node to the commit and shutdown. It will require knowledge of all the material so far covered but it shouldn't be hard to follow, since everything needed has been covered almost completely. So, without further ado...

Creating the EmbeddedGraphDatabase

The first thing to do to start working with a Neo instance is to create a new EmbeddedGraphDatabase. This is a thin shell over an EmbeddedGraphDbImpl, so we instantiate that. We provide it with the user-provided parameter map (or an empty one) and the default implementations for factories for LockManager that will create LockManager instances which will use a provided TransactionManager, IdGenerator for assigning ids to created entities by delegating to the store managers, RelationshipTypeCreator for creating RelationshipTypes, TxIdGenerator for assigning ids to txs that delegates to the TransactionManager, TxFinishHook for creating tx synchonization hooks on tx finish that does nothing and LastCommittedTxIdSetter for storing the last successfully committed tx id that also does nothing. EmbeddedGraphDbImpl in turn creates a new TxModule to hold a TransactionManager and a XaDataSourceManager, a new LockManager, a new Config, and a GraphDbInstance (as of late there is also an Indexer but we conveniently ignore that).

The Config object has a long story ahead of it. It accepts and stores most of the instances created by the factories at EmbeddedGraphDbImpl and also creates a new IdGeneratorModule, a new PersistenceModule, an AdaptiveCacheManager and a GraphDbModule. That last one parses the parameters map and decides on what type of Cache to ask NodeManager to create and whether the user has requested a full r/w database or a read-only one, creating as a result - wait for it - yes, the proper NodeManager instance, though that work will actually happen later on, in GraphDbInstance. References to all these are stored and the whole GraphDbModule is kept in the Config.

GraphDbInstance is where the database store is started. On start(), it uses an AutoConfigurator, which in its very brief lifetime computes some sensible defaults for the sizes in-memory images of the various stores, regardless of whether they will be buffered or memory mapped. These sizes are also placed in the Config object. Next comes what we all have been waiting for - that's right, the Store instantiation. The TxModule is retrieved from the Config and is asked to registerDataSource() as the DEFAULT_DATA_SOURCE_NAME the NIO_NEO_DB_CLASS which currently is NeoStoreXaDataSource. The XaDataSourceManager in TxModule is passed the parameters and instantiates the class via reflection, assuming that there is a constructor accepting a Map argument (which represents the configuration as a parameter map) and stores the result in a Map, pointed to by the resource's name. As we have seen previously, NeoStoreXaDataSource creates the actual store via instantiating a NeoStore and create()ing a XaContainer, possibly triggering a recovery or else "simply" instantiating the various datafiles and id stores. This is the major performance hiccup in the startup sequence, since all the above run in the same thread, a necessary measure to ensure a successful log recovery if it proves necessary. Obviously, if the database was created, from now on you can see the files in the db directory.

Going back to GraphDbInstance, a NioNeoDbPersistenceSource is created and stored in the Config and also provided in the IdGeneratorModule, where it is used as the actual source of entity ids. Note that the actual association of the PersistenceSource with the DataSource is made when a few lines below, GraphDbInstance calls start on the PersistenceSource passing as an argument the XaDataSourceManager. After that, init() is called on the modules in the Config (which currently do nothing) and then they are start()ed. This makes the TxModule to bind the XaDataSourceManager to the TransactionManager, the PersistenceModule to create the PersistenceManager, the PersistenceSource to aquire the DataSource, the IdGenerator to get the PersistenceSource and the GraphDbModule to create and start() the NodeManager. Starting the NodeManager causes the parameter map to be parsed and discover the sizes and type of the caches and register them with the CacheManager.
So there we are. The store is initialized, as well as the DataSource over it, the TxModule is up and running with its TransactionManager, the NodeManager has built all that it needs and the LockReleaser and LockManager are in place. This is pretty much what is needed to start working, so it is about time we did that.

Beginning a transaction

Explicitly beginning a tx is necessary only if you need to perform write operations. For read-only scenarios, where no locks are acquired, you can get away by simply asking the db for the element. This is not the typical (or interesting for that matter) scenario, so let's create something. This requires a tx so let's start that. Calling beginTx() on an EmbeddedGraphDbImpl asks the referenced GraphDbInstance for the TransactionManager (stored in the TxModule in the Config) and then asking to begin() one. No reference needs to be stored, recall that txs are thread bound, so as long as we are in the same thread we know which is our tx. However, an API must be provided for the demarcation of transactional code, so a TopLevelTransaction object is created, wrapped around the TxManager and returned to the user. This object is a simple wrapper around the TxManager, forwarding all calls of the Transaction interface to it, relying on the thread-to-tx mapping stored in the TxManager for the operation success. That is the object you receive on beginTx() so that you can work your magic.
We have already seen in some detail the workings of the TxManager class (which is the implementation of the TransactionManager interface) but let's follow the code. Calling begin() retrieves the currentThread() and maps a new TransactionImpl to it. Creating the TxImpl also assigns it a global Tx identifier via the Xid mechanism. Note that for now, no resources are enlisted, no log entries have been written and the state of the tx is STATUS_ACTIVE. To see in action the full mechanism we have to create a Node.

Creating the Node

To create a Node we call createNode() on EmbeddedGraphDatabase which forwards it to EmbeddedGraphDbImpl which sends it to NodeManager.createNode(). There the IdGenerator is asked for the nextId() for Node objects, which hands it off to the PersistenceSource. The implementing class is NioNeoDbPersistenceSource, which forwards it to the NeoStoreXaDataSource, which finally retrieves the NodeStore and returns the nextId(). You must have noticed here that in fact there is no Resource enlisted in the current tx yet. Now that we have the id, we can create a new NodeImpl and acquire a WRITE lock on it, creating also a NodeProxy object to return to the user.
Now comes the fun part. Still in NodeManager.createNode(), we ask the PersistenceManager to nodeCreate() the brand new Node for this id. The PersistenceManager has no idea how to do that, so it getResource() to get a ResourceConnection to do it. Of course the ResourceConnection is returned by the referenced PersistenceSource instance (which in our case is a NioNeoDbPersistenceSource returning NioNeoDbResourceConnections) and it indeed has a reference to an XAResource (that is an inner class that simply holds an Object as an id). So, after retrieving the current tx it asks it to enlist the XAResource, leading to all the magic described here. Also, a TxCommitHook is added to the tx that releases the connection and the makes the resources used reclaimable by the garbage collector upon the end of the connection usable life. Note that the XaResource is registered with the XaResourceManager and mapped to a WriteTransaction when the resource is start()ed in the TransactionImpl.
After we write to the TxLog and setup the XAResource to the tx, we still have an operation to do. Recall that the Connection holds the EventConsumers which forward to the corresponding store. The related consumer in this case is the NeoStoreXaConnection.NodeEventConsumerImpl, which for createNode() events (and all others for that matter) retrieves the WriteTransaction and tells it to nodeCreate() for the id and the WriteTransaction creates the NodeRecord object and stores it.
Let's make a check here: Nothing is written on the disk, including the Resource tx. The store is untouched, not even the id store has been tampered. The only thing in permanent storage is the global tx record marking it as STARTed and the branchId for the Resource. The fact that a record has been created is in memory only. If we crash here, there is an implicit rollback guarrantee.
We are not done yet, since the user has nothing to work with. Before returning the NodeProxy from the NodeManager, first we cache it and then we ask the LockReleaser (via NodeManager.releaseLock()) to release the Lock for this Node, an action that eventually results in keeping in memory the fact that this Node is locked and ask the current tx to add a Synchronization that will release the locks after completion. Now we can return the NodeProxy to the user.

Committing the transaction

So, the time has come to commit() the tx and make our valuable Node permanent. Calling success() on the TopLevelTransaction marks it simply as successful, meaning that on finish() it will be commit()ted. So, let's call finish(). The TransactionManager is asked for the current tx and commit() is called on it. This calls back the TransactionManager, which gets the current tx and does all those nice things that we discussed some time ago, such as writing to the TxLog and calling commit hooks. In a nutshell, the tx gets the single enlisted resource (since we are not using but one) and decides that, since this is a 1PC we just tell the resource to commit(). In our case, this leads to the NeoStoreXaConnection.NeoStoreXaResource to call the XaResourceManager to commit, which means that the WriteTransaction is asked to prepare(), compiling our new NodeRecord to a NodeCommand and writing that out to the XaLogicalLog, an action after which we can rest assured that our change will be performed no matter what. If this succeeds, XaResourceManager calls commit() on the WriteTransaction, meaning the Node creation command is executed, asking the NodeStore to write out that Record. We are now done, meaning the XaLogicalLog adds a DONE entry, the Transaction is removed from the txThreadMap in the TxManager, the TxLog marks the Transaction also as TX_DONE and the Transaction status is set to STATUS_NO_TRANSACTION. Now everything is on disk and both the Resource and the global txs are marked as successful.

Closing the database

We wrote our target 9 bytes on the disk (you do remember that a Node record is 9 bytes, right?) and we are ready to close the database. So we go ahead and call shutdown() on the EmbeddedGraphDatabase, which ends up calling GraphDbInstance.shutdown(). The config is asked for the various modules and tells them to stop(). The GraphDbModule tells the NodeManager to clearPropertyIndexes() and clearCache() and then stop(), operations that do nothing fancy, they just null out various references. The IdGeneratorModule and PersistenceModule have no-op stop() methods. The TxModule.stop() asks the TxManager to close() the TxLog that in turn close()es the log file channel. The most interesting part is in PersistenceSource.stop(). This forwards to NeoStoreXaDataSource which calls flushAll() to NeoStore, leading eventually to every store force()ing all buffers out to disk. This ensures a nice, clean shutdown of the disk store. The XaLogicalLog must also be closed, an operation already described in detail <a>some time before</a>. Then we call close() on NeoStore, which in essence closes the file channels used for persistence operations. This ends the file channel closing cycle, leaving the GraphDbImpl to call destroy() on the same modules, which currently are all no-ops. We can now exit.

From here

This post concludes a rough description of the core operations in Neo. I estimate around 2/3 of the code in the kernel component have been covered, leaving out aspects such as kernel component registrations, the indexing framework, traversals and a host of other extremely interesting components. I do not know to what extent I could have fitted them here, but I think that by understanding what I have discussed so far, one can navigate the code and understand the remaining pieces easily.
Truth been told, I have reached a point where I no longer want to write about Neo but instead I want to start hacking it. If the need arises and I find it interesting, I may write again about some other operation, but first I want to get a better feeling for the code. If you have specific requests/questions regarding my articles, I suggest you send a mail to the Neo mailing list and we can discuss it there, or hang around the #neo4j channel at freenode.

I hope that by reading my work you got at least part of the knowledge I got out of writing it.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.

No comments:

Post a Comment