Deadlocks and indexes in the ActiveMemory
One of least known, but most important, aspects of the ActiveMemory is its support for indexes. The first, but actually less important, reason to use indexes is performance: As in conventional databases, indexes are essential for good performance. Due to the simple nature of the XPath expressions used in AM queries, performance is still OK without them, particularly when the Memory has only a few entries -- as is usually the case when people test their components, so this often goes unnoticed.
However, the more important reason to use indexes is that, without indexes, deadlock potential is much higher. The reason for this is simple: When removing all content that matches an XPath, DBXML creates a candidate set of documents which need to be parsed to determine a match.For all of them, an "advisory lock" is set, which announces the intention to delete something, after the scan has completed.
Such locks not only prevent modification of a document, they also prevent modification of the part of the btree that stores database information about that documen. These data-structures are shared with other documents, and, most importantly, are modified when a new document is inserted. Thus, the first gotcha is that a lock can affect much more than just the document it applies to.
The second gotcha is that, if there are no indexes, the candidate set is the whole memory! This often goes unnoticed when the memory is small, because the scan is fast. However, once the memory grows, scanning takes much longer and deadlocks often result.
Therefore, use indexes! See https://code.ai.techfak.uni-bielefeld.de/trac/xcf/wiki/ActiveMemoryIndexing for more information on how to set them. Note that the indexes must match your remove queries in particular. Other queries are not as important, but indexes benefit them too, by speeding them up orders of magnitudes!
However, the more important reason to use indexes is that, without indexes, deadlock potential is much higher. The reason for this is simple: When removing all content that matches an XPath, DBXML creates a candidate set of documents which need to be parsed to determine a match.For all of them, an "advisory lock" is set, which announces the intention to delete something, after the scan has completed.
Such locks not only prevent modification of a document, they also prevent modification of the part of the btree that stores database information about that documen. These data-structures are shared with other documents, and, most importantly, are modified when a new document is inserted. Thus, the first gotcha is that a lock can affect much more than just the document it applies to.
The second gotcha is that, if there are no indexes, the candidate set is the whole memory! This often goes unnoticed when the memory is small, because the scan is fast. However, once the memory grows, scanning takes much longer and deadlocks often result.
Therefore, use indexes! See https://code.ai.techfak.uni-bielefeld.de/trac/xcf/wiki/ActiveMemoryIndexing for more information on how to set them. Note that the indexes must match your remove queries in particular. Other queries are not as important, but indexes benefit them too, by speeding them up orders of magnitudes!
- ingo's blog
- Log in to post comments
Recent Photos
