187 lines
6 KiB
Plaintext
187 lines
6 KiB
Plaintext
|
To-Do List for LKRhash
|
||
|
|
======================
|
||
|
|
|
||
|
|
TESTING
|
||
|
|
=======
|
||
|
|
|
||
|
|
* Write a C test, to directly test the C API. HashTest half does
|
||
|
|
it, as it can use the public API and TypedHashTable.
|
||
|
|
|
||
|
|
* Build a publicly distributable test program, as a sample for
|
||
|
|
customers. E.g., a dictionary component for ASP.
|
||
|
|
|
||
|
|
* hashtest: modify so that each thread can work on a completely
|
||
|
|
separate hashtable, instead of sharing a global
|
||
|
|
hashtable. This will allow us to discover the maximum possible
|
||
|
|
scaling, which is probably less than the `ideal' scaling,
|
||
|
|
#CPUs * 1-thread-perf.
|
||
|
|
|
||
|
|
* hashtest\kernel: split into driver and usermode test
|
||
|
|
program. Usermode test program should take care of parsing the
|
||
|
|
arguments and loading the test data into memory, then use an
|
||
|
|
ioctl to pass it down to the kernel driver. The only code in
|
||
|
|
the kernel driver,apart from LKRhash itself, should be the goo
|
||
|
|
to crack the ioctl and DriverEntry. Usermode should print
|
||
|
|
results.
|
||
|
|
|
||
|
|
* Better tests for ApplyIf family
|
||
|
|
|
||
|
|
|
||
|
|
KERNEL-MODE
|
||
|
|
===========
|
||
|
|
|
||
|
|
* kernel locks: should they block at all? consider implications
|
||
|
|
if running on some usermode program's thread.
|
||
|
|
|
||
|
|
* Fix global objects in driver, including the global list of
|
||
|
|
tables
|
||
|
|
|
||
|
|
* implement kernel-mode version of !lkrhash
|
||
|
|
|
||
|
|
* Think about running at DISPATCH_LEVEL instead of
|
||
|
|
PASSIVE_LEVEL. Which functions should be pageable
|
||
|
|
vs. non-pageable? Memory allocators? Lock types?
|
||
|
|
|
||
|
|
* Use <zwapi.h>?
|
||
|
|
|
||
|
|
* Memory allocation pool is a parameter for LKRhash
|
||
|
|
public constructors, but it's ignored. Need virtual base for
|
||
|
|
allocators?
|
||
|
|
|
||
|
|
* DONE: Control debug spew: provide some way of setting
|
||
|
|
g_fDebugOutputEnabled
|
||
|
|
|
||
|
|
|
||
|
|
LOCK IMPLEMENTATION
|
||
|
|
===================
|
||
|
|
|
||
|
|
* WriteLock: make all return a value instead of `void' that's
|
||
|
|
passed into WriteUnlock. Ditto for ReadLock, etc. Needed for
|
||
|
|
CKSpinLock and OldIrql. <sigh/>
|
||
|
|
|
||
|
|
* Put delays into the inner loop of the spins, to see if that
|
||
|
|
reduces cache sloshing
|
||
|
|
|
||
|
|
* Enable per-class instrumentation for locks, as opposed to the
|
||
|
|
current all-or-nothing system for per-lock
|
||
|
|
instrumentation. Mostly present already, just needs a little
|
||
|
|
factoring.
|
||
|
|
|
||
|
|
* Locks code. Move all member function implementations and enums
|
||
|
|
into locks.cpp. Locks.h should be opaque declarations only.
|
||
|
|
|
||
|
|
* Rewrite a few key functions, such as CSmallSpinLock::WriteLock
|
||
|
|
or CReaderWriterLock3::ReadOrWriteLock, in x86 assembler
|
||
|
|
|
||
|
|
* Build locks as a separate statically linked library
|
||
|
|
|
||
|
|
* Refcount the initialization/cleanup routines?
|
||
|
|
|
||
|
|
* Move CKSpinLock, CEResource, and CFastMutex into <Locks.h>
|
||
|
|
|
||
|
|
* DONE: Sprinkle KeEnterCriticalRegion (and KeLeaveCriticalRegion)
|
||
|
|
in the various locks, to prevent APCs being delivered that might
|
||
|
|
suspend the thread that holds the lock.
|
||
|
|
|
||
|
|
|
||
|
|
MISCELLANEOUS
|
||
|
|
=============
|
||
|
|
|
||
|
|
* Reduce the three versions of LKRhash in IIS6 to just this one.
|
||
|
|
|
||
|
|
* Currently, we key the number of subtables off LK_TABLESIZE
|
||
|
|
times a function of the number of CPUs. However, we only need
|
||
|
|
a lot of subtables under two circumstances: (a) many millions
|
||
|
|
of elements (esp. on Win64, where total number of elements
|
||
|
|
might exceed 2^32), or (b) high contention. There isn't
|
||
|
|
necessarily a correlation between large table size and high
|
||
|
|
contention. With the multi-reader locks, high contention only
|
||
|
|
arises if there are a lot of insertions/deletions. If the
|
||
|
|
table is not modified much after it's built, contention
|
||
|
|
shouldn't be an issue and there's little advantage to having a
|
||
|
|
lot of subtables.
|
||
|
|
|
||
|
|
* Provide a DeleteKey variant that returns a pointer to the
|
||
|
|
record that is being removed from the table.
|
||
|
|
|
||
|
|
* Make AddRefRecord return the new reference count.
|
||
|
|
|
||
|
|
* Inline the array of pointers to subtables within
|
||
|
|
CLKRHashTable, instead of dynamically allocating it.
|
||
|
|
|
||
|
|
* Inline _FindBucket by hand into Delete(Key|Record),
|
||
|
|
InsertRecord, and Find(Key|Record).
|
||
|
|
|
||
|
|
* Factor out memory stuff into LKR-mem.cpp
|
||
|
|
|
||
|
|
* Add some instrumentation: #allocs, #expands, #contracts, etc.
|
||
|
|
|
||
|
|
* Add some flags to LKR_Initialize: default size, output
|
||
|
|
tracing, etc
|
||
|
|
|
||
|
|
* Should ApplyIf(LKP_DELETE) call the Action function before
|
||
|
|
deleting? Or add LKP_PERFORM_DELETE[_STOP]
|
||
|
|
|
||
|
|
* Add debug print routines for the other enumerations and for
|
||
|
|
lock state to LKRhash.
|
||
|
|
|
||
|
|
* Experiment with having no bucket locks whatsoever, just
|
||
|
|
subtable locks. This should make operations a little faster,
|
||
|
|
but presumably will hurt multiprocessor scalability
|
||
|
|
|
||
|
|
* Implement fMultiKeys to provide support for multiple,
|
||
|
|
identical keys. Needed for EqualRange, hash_multiset, and
|
||
|
|
hash_multimap. See CLKRLinearHashTable::_InsertRecord for
|
||
|
|
details on what needs to be changed.
|
||
|
|
|
||
|
|
* Provide implementations of the STL collection classes:
|
||
|
|
hash_map, hash_set, hash_multimap, and hash_multiset. Must
|
||
|
|
provide full implementation of STL iterator semantics
|
||
|
|
(pair<key,value>)
|
||
|
|
|
||
|
|
* const_iterators for STL-style iterators
|
||
|
|
|
||
|
|
* consider providing some kind of implicit locking with
|
||
|
|
STL-style iterators
|
||
|
|
|
||
|
|
* Add version.subversion number to CLKRLinearHashTable
|
||
|
|
|
||
|
|
* Build lkrhash as a separate statically linked library
|
||
|
|
|
||
|
|
* Better Statistics: #buckets, density, by subtable
|
||
|
|
|
||
|
|
* Experiment with new hash function from Paul Larson.
|
||
|
|
|
||
|
|
* Public API in LKR-hash.h: remove dependency on irtlmisc.h and
|
||
|
|
irtldbg.h
|
||
|
|
|
||
|
|
* Use IsBadCodePtr to validate callback functions
|
||
|
|
|
||
|
|
* Make exception-safe. What happens if callback routines
|
||
|
|
(AddRefRecord/ExtractKeys or ApplyIf) access violate (throw an
|
||
|
|
SEH exception) or throw a C++ exception? Table and bucket
|
||
|
|
locks won't be released, state variables may be corrupted,
|
||
|
|
etc. LKRhash code should never throw any kind of exception
|
||
|
|
|
||
|
|
* Add some kind of auto object for readlocking or writelocking a
|
||
|
|
table, so that the table automatically gets unlocked by
|
||
|
|
auto-obj's destructor.
|
||
|
|
|
||
|
|
* Use auto_ptrs.
|
||
|
|
|
||
|
|
* Port to C# (Chris Tracy has started on this).
|
||
|
|
Andre Rosenthal has started a port to Managed C++
|
||
|
|
|
||
|
|
* Make LKRhash available as a static library as well as a DLL
|
||
|
|
|
||
|
|
* DONE: break apart lkrhash.cpp: iterators, applyif, stats, etc
|
||
|
|
|
||
|
|
* DONE: Always step forward through all subtables, to avoid
|
||
|
|
possible deadlock, when acquiring subtable locks
|
||
|
|
|
||
|
|
* DONE: Test new contraction algorithm.
|
||
|
|
|
||
|
|
* DONE: sublocks for ApplyIf
|
||
|
|
|
||
|
|
* DONE: Provide a C API wrapper
|