API¶

Serializers¶

KT_BINARY¶: Default value serialization. Serializes values as UTF-8 byte-strings and deserializes to unicode.

KT_JSON¶: Serialize values as JSON (encoded as UTF-8).

KT_MSGPACK¶: Uses msgpack to serialize and deserialize values.

KT_NONE¶: No serialization or deserialization. Values must be byte-strings.

KT_PICKLE¶: Serialize and deserialize using Python’s pickle module.

Kyoto Tycoon client¶

class KyotoTycoon(host='127.0.0.1', port=1978, timeout=None, default_db=0, decode_keys=True, serializer=None, encode_value=None, decode_value=None, max_age=3600)¶

Parameters:

host (str) – server host.
port (int) – server port.
timeout (int) – socket timeout for database connection.
default_db (int) – default database index.
decode_keys (bool) – decode keys as utf8-encoded unicode.
serializer – serialization method to use for storing/retrieving values. Default is KT_BINARY, which treats values as utf8-encoded unicode. KT_NONE disables all serialization, or use one of KT_JSON, KT_MSGPACK or KT_PICKLE.
encode_value – custom serializer for encoding values as bytestrings.
decode_value – custom deserializer for decoding bytestrings.
max_age (int) – max idle time for socket in connection pool.

Client for interacting with Kyoto Tycoon database.

set_database(db)¶

Parameters:: db (int) – database index.

Set the default database for the client. The Kyoto Tycoon server can be run by specifying multiple database paths. This method allows you to specify which database the client communicates with by default, though most methods accept a db parameter which can override the default for a given call.

close_all()¶

Returns:: number of connections that were closed.

Close all connections in the connection pool. The pool maintains two sets of connections:

Binary protocol connections.
HTTP client connections for the HTTP API.

Since the binary protocol only implements a subset of the total commands, ukt will transparently use the appropriate connection type for a given method.

serialize_dict(d, encode_values=True)¶

Parameters:

d (dict) – arbitrary data.
encode_values (bool) – serialize the values using the configured serialization scheme.

Returns:

serialized data.

Serialize a dict as a sequence of bytes compatible with KT’s built-in lua mapdump function and the Hash container type.

deserialize_dict(data, decode_values=True)¶

Parameters:

data (bytes) – serialized data.
decode_values (bool) – decode values using the configured serialization scheme.

Returns:

data dict.

Deserialize a sequence of bytes into a dictionary, optionally decoding the values as unicode strings. Compatible with KT’s built-in lua mapload function and the Hash container type.

serialize_list(l, encode_values=True)¶

Parameters:

l (list) – arbitrary data.
encode_values (bool) – serialize the values using the configured serialization scheme.

Returns:

serialized data.

Serialize a list as a sequence of bytes compatible with KT’s built-in lua arraydump function and the List container type.

deserialize_list(data, decode_values=True)¶

Parameters:

data (bytes) – serialized data.
decode_values (bool) – decode values using the configured serialization scheme.

Returns:

data list.

Deserialize a a sequence of bytes into a list, optionally decoding the values as unicode strings. Compatible with KT’s built-in lua arrayload function and the List container type.

get_bulk(keys, db=None, decode_values=True)¶

Parameters:

keys (list) – keys to retrieve.
db (int) – database index.
decode_values (bool) – decode values using the configured serialization scheme.

Returns:

result dictionary

Efficiently retrieve multiple key/value pairs from a database. If a key does not exist, it will not be present in the result dictionary.

get_bulk_details(db_key_list, decode_values=True)¶

Parameters:

db_key_list (list) – a list of (db, key) tuples to fetch.
decode_values (bool) – decode values using the configured serialization scheme.

Returns:

list of tuples: (db index, key, value, expire time)

Like get_bulk(), but the return value is a list of tuples with additional information for each key. Since each key is of the form (db, key), this method can be used to efficiently fetch records from multiple databases.

get(key, db=None, decode_value=True)¶

Parameters:

key (str) – key to look-up
db (int) – database index
decode_value (bool) – decode value using serializer.

Returns:

deserialized value or None if key does not exist.

Fetch and (optionally) deserialize the value for the given key.

get_bytes(key, db=None)¶

Parameters:

key (str) – key to look-up
db (int) – database index

Returns:

raw bytestring value or None if key does not exist.

Fetch the value for the given key. The resulting value will not be deserialized.

set_bulk(data, db=None, expire_time=None, no_reply=False, encode_values=True)¶

Parameters:

data (dict) – mapping of key/value pairs to set.
db (int) – database index
expire_time (int) – expiration time in seconds
no_reply (bool) – execute the operation without a server acknowledgment.
encode_values (bool) – serialize the values using the configured serialization scheme (e.g., KT_MSGPACK).

Returns:

number of keys that were set, or None if no_reply.

Efficiently set multiple key/value pairs. If given, the provided db and expire_time values will be used for all key/value pairs being set.

set_bulk_details(data, no_reply=False, encode_values=True)¶

Parameters:

data (list) – a list of 4-tuples: (db, key, value, expire-time)
no_reply (bool) – execute the operation without a server acknowledgment.
encode_values (bool) – serialize the values using the configured serialization scheme (e.g., KT_MSGPACK).

Returns:

number of keys that were set, or None if no_reply.

Efficiently set multiple key/value pairs. Unlike set_bulk(), this method can be used to set key/value pairs in multiple databases in a single call, and each key can specify its own expire time.

set(key, value, db=None, expire_time=None, no_reply=False, encode_value=True)¶

Parameters:

key (str) – key to set.
value – value to store.
db (int) – database index.
expire_time (int) – expiration time in seconds.
no_reply (bool) – execute the operation without a server acknowledgment.
encode_value (bool) – encode value using serializer.

Returns:

number of rows set (1)

Set a single key/value pair.

set_bytes(key, value, db=None, expire_time=None, no_reply=False)¶

Parameters:

key (str) – key to set.
value (bytes) – raw bytes to store.
db (int) – database index.
expire_time (int) – expiration time in seconds.
no_reply (bool) – execute the operation without a server acknowledgment.

Returns:

number of rows set (1)

Set a single key/value pair, without serializing the value.

remove_bulk(keys, db=None, no_reply=False)¶

Parameters:

keys (list) – list of keys to remove
db (int) – database index
no_reply (bool) – execute the operation without a server acknowledgment.

Returns:

number of keys that were removed

Remove multiple keys from a database in a single operation.

remove_bulk_details(db_key_list, no_reply=False)¶

Parameters:

db_key_list – a list of 2-tuples to retrieve: (db index, key)
no_reply (bool) – execute the operation without a server acknowledgment.

Returns:

number of keys that were removed

Like remove_bulk(), but allows keys to be removed from multiple databases in a single call. The input is a list of (db, key) tuples.

remove(key, db=None, no_reply=False)¶

Parameters:

key (str) – key to remove
db (int) – database index
no_reply (bool) – execute the operation without a server acknowledgment.

Returns:

number of rows removed

Remove a single key from the database.

script(name, data=None, no_reply=False, encode_values=True, decode_values=True)¶

Parameters:

name (str) – name of lua function to call.
data (dict) – mapping of key/value pairs to pass to lua function.
no_reply (bool) – execute the operation without a server acknowledgment.
encode_values (bool) – serialize values passed to lua function.
decode_values (bool) – deserialize values returned by lua function.

Returns:

dictionary of key/value pairs returned by function.

Execute a lua function. Kyoto Tycoon lua extensions accept arbitrary key/value pairs as input, and return a result dictionary. If encode_values is True, the input values will be serialized. Likewise, if decode_values is True the values returned by the Lua function will be deserialized using the configured serializer.

raw_script(name, data=None, no_reply=False)¶

Parameters:

name (str) – name of lua function to call.
data (dict) – mapping of key/value pairs to pass to lua function.
no_reply (bool) – execute the operation without a server acknowledgment.

Returns:

dictionary of key/value pairs returned by function.

Execute a lua function and return the result with no post-processing or serialization.

report()¶

Returns:: status fields and values
Return type:: dict

Obtain report on overall status of server, including all databases.

status(db=None)¶

Parameters:: db (int) – database index
Returns:: status fields and values
Return type:: dict

Obtain status information from the server about the selected database.

list_databases()¶

Returns:: a list of (database path, status dict) for each configured database.

Return the list of databases and their status information.

databases¶: Returns the list of paths for the configured databases.

clear(db=None)¶

Parameters:: db (int) – database index
Returns:: boolean indicating success

Remove all keys from the database.

synchronize(hard=False, command=None, db=None)¶

Parameters:

hard (bool) – perform a “hard” synchronization.
command (str) – command to execute after synchronization.
db (int) – database index.

Returns:

boolean indicating success.

Synchronize the database, optionally executing the given command upon success. This can be used to create hot backups, for example.

add(key, value, db=None, expire_time=None, encode_value=True)¶

Parameters:

key (str) – key to add.
value – value to store.
db (int) – database index.
expire_time (int) – expiration time in seconds.
encode_value (bool) – serialize the value using the configured serialization method.

Returns:

boolean indicating if key could be added or not.

Return type:

bool

Add a key/value pair to the database. This operation will only succeed if the key does not already exist in the database.

replace(key, value, db=None, expire_time=None, encode_value=True)¶

Parameters:

key (str) – key to replace.
value – value to store.
db (int) – database index.
expire_time (int) – expiration time in seconds.
encode_value (bool) – serialize the value using the configured serialization method.

Returns:

boolean indicating if key could be replaced or not.

Return type:

bool

Replace a key/value pair to the database. This operation will only succeed if the key alreadys exist in the database.

append(key, value, db=None, expire_time=None, encode_value=True)¶

Parameters:

key (str) – key to append value to.
value – data to append.
db (int) – database index.
expire_time (int) – expiration time in seconds.
encode_value (bool) – serialize the value using the configured serialization method.

Returns:

boolean indicating if value was appended.

Return type:

bool

Appends data to an existing key/value pair. If the key does not exist, this is equivalent to set().

increment(key, n=1, orig=None, db=None, expire_time=None)¶

Parameters:

key (str) – key to increment.
n (int) – value to add.
orig (int) – default value if key does not exist.
db (int) – database index.
expire_time (int) – expiration time in seconds.

Returns:

new value at key.

Return type:

int

Increment the value stored in the given key.

increment_double(key, n=1., orig=None, db=None, expire_time=None)¶

Parameters:

key (str) – key to increment.
n (float) – value to add.
orig (float) – default value if key does not exist.
db (int) – database index.
expire_time (int) – expiration time in seconds.

Returns:

new value at key.

Return type:

float

Increment the floating-point value stored in the given key.

cas(key, old_val, new_val, db=None, expire_time=None, encode_value=True)¶

Parameters:

key (str) – key to append value to.
old_val – original value to test.
new_val – new value to store.
db (int) – database index.
expire_time (int) – expiration time in seconds.
encode_value (bool) – serialize the old and new values using the configured serialization method.

Returns:

boolean indicating if compare-and-swap succeeded.

Return type:

bool

Perform an atomic compare-and-set the value stored at a given key.

exists(key, db=None)¶

Parameters:

key (str) – key to test.
db (int) – database index.

Returns:

boolean indicating if key exists.

Return whether or not the given key exists in the database.

length(key, db=None)¶

Parameters:

key (str) – key.
db (int) – database index.

Returns:

length of the value in bytes, or None if not found.

Return the length of the raw value stored at the given key. If the key does not exist, returns None.

seize(key, db=None, decode_value=True)¶

Parameters:

key (str) – key to remove.
db (int) – database index.
decode_value (bool) – deserialize the value using the configured serialization method.

Returns:

value stored at given key or None if key does not exist.

Perform atomic get-and-remove the value stored in a given key. This method is also available as KyotoTycoon.pop() if that’s easier to remember.

vacuum(step=0, db=None)¶

Parameters:

step (int) – number of steps, default is 0
db (int) – database index

Returns:

boolean indicating success

Vacuum the database.

match_prefix(prefix, max_keys=None, db=None)¶

Parameters:

prefix (str) – key prefix to match.
max_keys (int) – maximum number of results to return (optional).
db (int) – database index.

Returns:

list of keys that matched the given prefix.

Return type:

list

Return sorted list of keys that match the given prefix.

match_regex(regex, max_keys=None, db=None)¶

Parameters:

regex (str) – regular-expression to match
max_keys (int) – maximum number of results to return (optional)
db (int) – database index

Returns:

list of keys that matched the given regular expression.

Return type:

list

Return sorted list of keys that match the given regular expression.

match_similar(origin, distance=None, max_keys=None, db=None)¶

Parameters:

origin (str) – source string for comparison
distance (int) – maximum edit-distance for similarity (optional)
max_keys (int) – maximum number of results to return (optional)
db (int) – database index

Returns:

list of keys that were within a certain edit-distance of origin

Return type:

list

Return sorted list of keys that are within a given edit distance from a string.

ulog_list()¶

Returns:: a list of 3-tuples describing the files in the update log.

Returns a list of metadata about the state of the update log. For each file in the update log, a 3-tuple is returned. For example:

>>> kt.ulog_list()
[('/var/lib/database/ulog/kt/0000000037.ulog',
  '67150706',
  datetime.datetime(2019, 1, 4, 1, 28, 42, 43000)),
 ('/var/lib/database/ulog/kt/0000000038.ulog',
  '14577366',
  datetime.datetime(2019, 1, 4, 1, 41, 7, 245000))]

ulog_remove(max_dt)¶

Parameters:: max_dt (datetime) – maximum datetime to preserve
Returns:: boolean indicating success

Removes all update-log files older than the given datetime.

count(db=None)¶

Parameters:: db (int or None) – database index
Returns:: total number of keys in the database.
Return type:: int

Count total number of keys in the database.

size(db=None)¶

Parameters:: db (int or None) – database index
Returns:: size of database in bytes.

Property which exposes the size information returned by the status() API.

__getitem__(key_or_keydb)¶: Item-lookup based on either key or a 2-tuple consisting of (key, db). Follows same semantics as get().

__setitem__(key_or_keydb, value_or_valueexpire)¶: Item-setting based on either key or a 2-tuple consisting of (key, db). Value consists of either a value or a 2-tuple consisting of (value, expire_time). Follows same semantics as set().

__delitem__(key_or_keydb)¶: Item-deletion based on either key or a 2-tuple consisting of (key, db). Follows same semantics as remove().

__contains__(key_or_keydb)¶: Check if key exists. Accepts either key or a 2-tuple consisting of (key, db). Follows same semantics as exists().

__len__()¶

Returns:: total number of keys in the default database.
Return type:: int

update(__data=None, **kwargs)¶

Parameters:

__data (dict) – optionally provide data as a dictionary.
kwargs – provide data as keyword arguments.

Returns:

number of keys that were set.

Efficiently set or update multiple key/value pairs. Provided for compatibility with dict interface. For more control use the set_bulk().

pop(key, db=None, decode_value=True)¶

Get and remove the data stored in a given key in a single operation.

See KyotoTycoon.seize().

keys(db=None)¶

Parameters:: db (int) – database index
Returns:: all keys in database
Return type:: generator

Warning

The keys() method uses a cursor and can be very slow.

keys_nonlazy(db=None)¶

Parameters:: db (int) – database index
Returns:: all keys in database
Return type:: list

Non-lazy implementation of keys(). Behind-the-scenes, calls match_prefix() with an empty string as the prefix.

values(db=None)¶

Parameters:: db (int) – database index
Returns:: all values in database
Return type:: generator

items(db=None)¶

Parameters:: db (int) – database index
Returns:: all key/value tuples in database
Return type:: generator

__iter__()¶: Iterating over the database yields an iterator over the keys of the database. Equivalent to keys().

touch(key, xt=None, db=None)¶

Parameters:

key (str) – key to update.
xt (int) – new expire time (or None).
db (int) – database index.

Returns:

old expire time or None if key not found.

Run a lua function (touch) defined in scripts/kt.lua that allows one to update the TTL / expire time of a key.

The old expire time is returned. If the key does not exist, then None is returned.

touch_bulk(keys, xt=None, db=None)¶

Parameters:

keys (list) – keys to update.
xt (int) – new expire time (or None).
db (int) – database index.

Returns:

a dict of key -> old expire time.

Run a lua function (touch_bulk) defined in scripts/kt.lua that allows one to update the TTL / expire time of multiple keys.

The return value is a dictionary of key -> old expire time. If the key does not exist, then the key is omitted from the return value.

touch_relative(key, n, db=None)¶

Parameters:

key (str) – key to update.
n (int) – seconds to increase expire-time.
db (int) – database index.

Returns:

new expire time or None if key not found.

Run a lua function (touch_bulk_relative) defined in scripts/kt.lua that allows one to increment the TTL / expire time of a key.

The new expire time is returned. If the key does not exist, then None is returned.

touch_bulk_relative(keys, n, db=None)¶

Parameters:

keys (list) – keys to update.
n (int) – seconds to increase expire-time.
db (int) – database index.

Returns:

a dict of key -> new expire time.

Run a lua function (touch_bulk_relative) defined in scripts/kt.lua that allows one to update the TTL / expire time of multiple keys.

The return value is a dictionary of key -> new expire time. If the key does not exist, then the key is omitted from the return value.

expire_time(key, db=None)¶

Parameters:

key (str) – key to check.
db (int) – database index

Returns:

expire timestamp or None if key not found.

Get the expire time by running a lua function (expire_time) defined in scripts/kt.lua.

expires(key, db=None)¶

Parameters:

key (str) – key to check.
db (int) – database index

Returns:

expire datetime or None if key not found.

Get the expire time as a datetime.

error(db=None)¶

Parameters:: db (int) – database index.
Returns:: a 2-tuple of (code, message)

Get the last error code and message.

If the last command was successful, then (0, ‘success’) is returned.

Hash(key, encode_values=True, decode_values=True, db=None)¶

Parameters:

key (str) – key to store the hash table.
encode_values (bool) – serialize the hash values using the configured serializer.
decode_values (bool) – de-serialize the hash values using the configured serializer.
db (int) – database index.

Create a Hash container instance.

List(key, encode_values=True, decode_values=True, db=None)¶

Parameters:

key (str) – key to store the list.
encode_values (bool) – serialize the list items using the configured serializer.
decode_values (bool) – de-serialize the list items using the configured serializer.
db (int) – database index.

Create a List container instance.

Set(key, encode_values=True, decode_values=True, db=None)¶

Parameters:

key (str) – key to store the set.
encode_values (bool) – serialize the set keys using the configured serializer.
decode_values (bool) – de-serialize the set keys using the configured serializer.
db (int) – database index.

Create a Set container instance.

Queue(key, db=None)¶

Parameters:

key (str) – key to use for the queue metadata.
db (int) – database index.

Create a Queue, which provides efficient operations for implementing a priority queue.

Schedule(key, db=None)¶

Parameters:

key (str) – key to use for the schedule metadata.
db (int) – database index.

Create a Schedule, which provides efficient operations for implementing a sorted schedule.

cursor(db=None, cursor_id=None)¶

Parameters:

db (int) – database index
cursor_id (int) – cursor id (will be automatically created if None)

Returns:

Cursor object

class Cursor(protocol, cursor_id, db=None, decode_values=True, encode_values=True)¶

Parameters:

protocol (KyotoTycoon) – client instance.
cursor_id (int) – cursor unique identifier.
db (int) – database index.
decode_values (bool) – decode values using client serializer when reading from the cursor.
encode_values (bool) – encode values using client serializer when writing to the cursor.

Create a helper for working with the database using the cursor interface.

jump(key=None)¶

Parameters:: key (str) – key to jump to or None.
Returns:: boolean indicating success.

Jump to the given key. If not provided, will jump to the first key in the database.

jump_back(key=None)¶

Parameters:: key (str) – key to jump backwards to or None.
Returns:: boolean indicating success.

Jump backwards to the given key. If not provided, will jump to the last key in the database.

step()¶

Returns:: boolean indicating success.

Step to the next key. Returns False when past the last key of the database.

step_back()¶

Returns:: boolean indicating success.

Step to the previous key. Returns False when past the first key of the database.

key(step=False)¶

Parameters:: step (bool) – step to next record after reading.
Returns:: key of the currently-selected record.

value(step=False)¶

Parameters:: step (bool) – step to next record after reading.
Returns:: value of the currently-selected record.

get(step=False)¶

Parameters:: step (bool) – step to next record after reading.
Returns:: (key, value) of the currently-selected record.

set_value(value, step=False, expire_time=None)¶

Parameters:

value – value to set
step (bool) – step to next record after writing.
expire_time (int) – optional expire time for record.

Returns:

boolean indicating success.

Set the value at the currently-selected record.

remove()¶

Returns:: boolean indicating success.

Remove the currently-selected record.

seize()¶

Returns:: (key, value) of the currently-selected record.

Get and remove the currently-selected record.

close()¶

Returns:: boolean indicating success.

Close the cursor.

class Queue(client, key, db=None)¶

Parameters:

client (KyotoTycoon) – client instance.
key (str) – key to store queue data.
db (int) – database index.

Priority queue implementation using lua functions (provided in the scripts/kt.lua module).

add(item, score=None)¶

Parameters:

item – item to add to queue.
score (int) – score (for priority support), higher values will be dequeued first. If not provided, defaults to 0.

Returns:

id of newly-added item.

extend(items, score=None)¶

Parameters:

items (list) – list of items to add to queue.
score (int) – score (for priority support), higher values will be dequeued first. If not provided, defaults to 0.

Returns:

number of items added to queue.

pop(n=1, min_score=None)¶

Parameters:

n (int) – number of items to remove from queue.
min_score (int) – minimum priority score. If not provided, all items will be considered regardless of score.

Returns:

either a single item or a list of items (depending on n).

Pop one or more items from the head of the queue.

rpop(n=1, min_score=None)¶

Parameters:

n (int) – number of items to remove from end of queue.
min_score (int) – minimum priority score. If not provided, all items will be considered regardless of score.

Returns:

either a single item or a list of items (depending on n).

Pop one or more items from the end of the queue.

bpop(timeout=None, min_score=None)¶

Parameters:

timeout (int) – seconds to block before giving up.
min_score (int) – minimum priority score. If not provided, all items will be considered regardless of score.

Returns:

item from the head of the queue, or if no items are added before the timeout, None is returned.

Pop an item from the queue, blocking if no items are available.

peek(n=1, min_score=None)¶

Parameters:

n (int) – number of items to read from queue.
min_score (int) – minimum priority score. If not provided, all items will be considered regardless of score.

Returns:

either a single item or a list of items (depending on n).

Read (without removing) one or more items from the head of the queue.

rpeek(n=1, min_score=None)¶

Parameters:

n (int) – number of items to read from end of queue.
min_score (int) – minimum priority score. If not provided, all items will be considered regardless of score.

Returns:

either a single item or a list of items (depending on n).

Read (without removing) one or more items from the end of the queue.

count()¶

Returns:: number of items in the queue.

remove(data, n=None, min_score=None)¶

Parameters:

data – value to remove from queue.
n (int) – max occurrences to remove.
min_score (int) – minimum priority score. If not provided, all items will be considered regardless of score.

Returns:

number of items removed.

Remove one or more items by value, starting from the head of the queue.

rremove(data, n=None, min_score=None)¶

Parameters:

data – value to remove from end of queue.
n (int) – max occurrences to remove.
min_score (int) – minimum priority score. If not provided, all items will be considered regardless of score.

Returns:

number of items removed.

Remove one or more items by value, starting from the end of the queue.

transfer(dest, n=1)¶

Parameters:

dest – destination queue key or Queue instance.
n (int) – number of items to transfer.

Returns:

either the item that was transferred or the list of items that was transferred, depending on n.

Transfer items from the head of the queue to the tail of the destination queue. Priority scores are preserved. If the source queue is empty, then either None or an empty list will be returned (depending on whether n=1).

set_priority(data, score, n=None)¶

Parameters:

data – value to remove from end of queue.
score (int) – new score for the item.
n (int) – max occurrences to update.

Update the priority of one or more items in the queue, by value.

clear()¶

Returns:: number of items in queue when cleared.

Remove all items from queue.

class Schedule(client, key, db=None)¶

Parameters:

client (KyotoTycoon) – client instance.
key (str) – key to store schedule data.
db (int) – database index.

Prioritized schedule implementation using lua functions (provided in the scripts/kt.lua module).

add(item, score=0)¶

Parameters:

item – add an item to the schedule.
score (int) – score (arrival time) of item.

Add an item to the schedule, with a given score / arrival time.

read(score=None, n=None)¶

Parameters:

score (int) – score threshold or arrival time
n (int) – maximum number of items to read.

Returns:

a list of items

Destructively read up-to n items from the schedule, whose item score is below the given score.

clear()¶: Clear the schedule, removing all items.

count()¶

Returns:: number of items in the schedule.

Return the number of items in the schedule.

items(n=None)¶

Parameters:: n (int) – limit the number of items to read.
Returns:: a list of up-to n items from the schedule.

Non-destructively read up-to n items from the schedule, in order of score.

Container types¶

Simple container types that emulate Python or Redis types, and rely on Kyoto Tycoon’s lua serialization helpers. Behind-the scenes, these types are using lua functions to read the entire value into a Lua table and write it back. Because the full data must be deserialized for reading, and re-serialized for writing, all operations are O(n).

These container types support transparent serialization using the configured serializer (KT_PICKLE, KT_MSGPACK, etc).

class Hash(kt, key, encode_values=True, decode_values=True, db=None)¶

Parameters:

kt (KyotoTycoon) – client
key (str) – key to store hash data
encode_values (bool) – values should be serialized using the configured serializer (e.g., KT_PICKLE, KT_MSGPACK, etc).
decode_values (bool) – values should be deserialized using the configured serializer.
db (int) – database index to store hash. If not specified, will use the default db configured for the kt client.

set_bulk(__data=None, **kwargs)¶

Parameters:

__data (dict) – provide data as a dictionary.
kwargs – or provide data keyword arguments.

Returns:

number of keys that were set.

Update the data stored in the hash.

get_bulk(keys)¶

Parameters:: keys – an iterable of keys to fetch.
Returns:: a dictionary of key/value pairs. If a requested key is not found, it is not included in the returned data.

remove_bulk(keys)¶

Parameters:: keys – an iterable of keys to remove.
Returns:: number of key/value pairs that were removed.

get_all()¶

Returns:: dictionary of all data stored in the hash

A more efficient implementation utilizes the Python implementation of the lua serializers. Use Hash.get_raw().

set(key, value)¶

Parameters:

key (str) – key to store
value – data

Set a single key/value pair in the hash. Returns number of records written (1).

setnx(key, value)¶

Parameters:

key (str) – key to store
value – data

Returns:

1 on success, 0 if key already exists.

Set a single key/value pair in the hash only if the key does not already exist.

get(key)¶

Parameters:: key (str) – key to fetch
Returns:: value, if key exists, or None.

remove(key)¶

Parameters:: key (str) – key to remove
Returns:: number of keys removed, 1 on success, 0 if key not found.

length()¶

Returns:: total number of keys in the hash.

contains(key)¶

Parameters:: key (str) – key to check
Returns:: boolean indicating whether the given key exists.

unpack(prefix=None)¶

Parameters:: prefix (str) – prefix for unpacked-keys
Returns:: number of keys that were written

Unpack the key/value pairs in the hash into top-level key/value pairs in the database, optionally prefixing the unpacked keys with the given prefix.

pack(start=None, stop=None, count=None)¶

Parameters:

start (str) – start key, or will be first key in the database
stop (str) – stop key, or will be last key in the database
count (int) – limit number of keys to pack

Returns:

number of keys that were packed

Pack a range of key/value pairs in the database into a hash.

pack_keys(key)¶

Parameters:: key (str) – destination key for List of keys.
Returns:: number of keys that were written to the list

Pack the keys of the hash into a List at the given key.

pack_values(key)¶

Parameters:: key (str) – destination key for List of values.
Returns:: number of values that were written to the list

Pack the values of the hash into a List at the given key.

__len__()¶: See length().

__contains__()¶: See contains().

__getitem__()¶: See get().

__setitem__()¶: See set().

__detitem__()¶: See remove().

update(__data=None, **kwargs)¶: See set_bulk().

get_raw()¶

Returns:: dictionary of all data stored in hash, or None if empty.

Utilize a more-efficient implementation for fetching all data stored in the hash. Rather than going through Lua, we read the raw value of the serialized hash, then deserialize it using an equivalent format to KT’s internal mapload format.

set_raw(d)¶

Parameters:: d (dict) – dictionary of all data to store in hash.

Utilize a more-efficient implementation for setting the data stored in the hash. Rather than going through Lua, we write the raw value of the serialized hash, using an equivalent format to KT’s internal mapdump format.

class List(kt, key, encode_values=True, decode_values=True, db=None)¶

Parameters:

kt (KyotoTycoon) – client
key (str) – key to store list data
encode_values (bool) – values should be serialized using the configured serializer (e.g., KT_PICKLE, KT_MSGPACK, etc).
decode_values (bool) – values should be deserialized using the configured serializer.
db (int) – database index to store list. If not specified, will use the default db configured for the kt client.

appendleft(value)¶

Parameters:: value – value to append to left-side (head) of list.
Returns:: length of list after operation.

appendright(value)¶

Parameters:: value – value to append to right-side (tail) of list.
Returns:: length of list after operation.

append(value)¶: Alias for appendright().

extend(values)¶

Parameters:: values – an iterable of values to add to the tail of the list.
Returns:: length of list after operation.

get_range(start=None, stop=None)¶

Parameters:

start (int) – start index (0 for first element)
stop (int) – stop index. Supports negative values.

Returns:

a list of items corresponding to the given range.

Slicing operation equivalent to Python’s list slice behavior. If the start or stop indices are out-of-bounds, the return value will be an empty list.

index(index)¶

Parameters:: index (int) – item index to fetch. Supports negative values.
Returns:: the value at the given index

Indexing operation equivalent to Python’s list item lookup. If the index is out-of-bounds, an IndexError will be raised.

insert(index, value)¶

Parameters:

index (int) – index at which new value should be inserted. Supports negative values.
value – value to insert

Returns:

length of list after operation

Insert an item into the list at the given index. If the index is out-of-bounds, an IndexError will be raised.

remove(index)¶

Parameters:: index (int) – item index to remove. Supports negative values.
Returns:: the value at the given index

Remove and return an item from the list by index. If the index is out-of-bounds, an IndexError will be raised.

remove_range(start=None, stop=None)¶

Parameters:

start (int) – start index to remove. Supports negative values.
stop (int) – stop index of range to remove. Supports negative values.

Returns:

length of list after operation

Remove a range of values by index.

popleft()¶

Returns:: item at head of list or None if list is empty.

popright()¶

Returns:: item at tail of list or None if list is empty.

pop(index=None)¶

Parameters:: index (int) – index to pop (optional), or None to remove the item at the tail of the list.
Returns:: item removed or None if list is empty or the index is out-of-bounds.

lpoprpush(dest=None)¶

Parameters:: dest – destination key (or List object). If unspecified, the destination will be the current list and the operation is equivalent to a rotation.
Returns:: item that was moved, if source is not empty. If source list is empty, an IndexError is raised.

Pop the item at the head of the current list and push it to the tail of the dest list.

rpoplpush(dest=None)¶

Parameters:: dest – destination key (or List object). If unspecified, the destination will be the current list and the operation is equivalent to a rotation.
Returns:: item that was moved, if source is not empty. If source list is empty, an IndexError is raised.

Pop the item at the tail of the current list and push it to the head of the dest list.

length()¶

Returns:: length of the list.

set(index, value)¶

Parameters:

index (int) – index to set. Supports negative values.
value – value to set at given index

Set the value at the given index. If the index is out-of-bounds, an IndexError will be raised.

find(value)¶

Parameters:: value – value to search for
Returns:: index of first occurrance of value starting from head of list.

rfind(value)¶

Parameters:: value – value to search for
Returns:: index of first occurrance of value starting from tail of list.

unpack(start=None, stop=None, prefix=None, fmt=None)¶

Parameters:

start (int) – start index of range to unpack
stop (int) – stop index of range to unpack
prefix (str) – prefix for output values
fmt (str) – lua format-string for index, e.g. ‘%08d’.

Unpack the items in the list into top-level keys in the database. The key will begin with the provided prefix, and optionally accepts a format-string for formatting the index.

pack(start=None, stop=None, count=None)¶

Parameters:

start (str) – start key, or will be first key in the database
stop (str) – stop key, or will be last key in the database
count (int) – limit number of keys to pack

Returns:

number of keys that were packed

Pack the values for a range of keys in the database into a list.

__len__()¶: See length().

__contains__()¶: See find().

__getitem__()¶: Supports item indexes or slices. See index() and get_range().

__setitem__()¶: See set().

__detitem__()¶: See remove().

get_raw()¶

Returns:: list of all data stored in list, or None if empty.

Utilize a more-efficient implementation for fetching all data stored in the list. Rather than going through Lua, we read the raw value of the serialized list, then deserialize it using an equivalent format to KT’s internal arrayload format.

set_raw(l)¶

Parameters:: l (list) – list of all data to store in list.

Utilize a more-efficient implementation for setting the data stored in the list. Rather than going through Lua, we write the raw value of the serialized list, using an equivalent format to KT’s internal arraydump format.

Embedded Servers¶

class EmbeddedServer(server='ktserver', host='127.0.0.1', port=None, database='*', serializer=None, server_args=None, quiet=False)¶

Parameters:

server (str) – path to ktserver executable.
host (str) – host to bind server on.
port (int) – port to use (optional).
database (str) – database filename, default is in-memory hash table.
serializer – serializer to use, e.g. KT_BINARY or KT_MSGPACK.
server_args (list) – additional command-line arguments for server
quiet (bool) – minimal logging and output.

Create a manager for running an embedded (sub-process) Kyoto Tycoon server. If the port is not specified, a random high port will be used.

Example:

>>> from kt import EmbeddedServer
>>> server = EmbeddedServer()
>>> server.run()
True
>>> client = server.client
>>> client.set('k1', 'v1')
1
>>> client.get('k1')
'v1'
>>> server.stop()
True

run()¶

Returns:: boolean indicating if server successfully started

Run ktserver in a sub-process.

stop()¶

Returns:: boolean indicating if server was stopped

Stop the running embedded server.

client¶: KyotoTycoon client bound to the embedded server.

API¶

Serializers¶

Kyoto Tycoon client¶

Container types¶

Embedded Servers¶

ukt

Navigation

Related Topics