9.4 Browsing Data with Key

In this section, we will learn multiple ways to browse the data stored in Drip. In Drip, all the elements are stored in the order they were added. Browsing data in Drip is like time traveling.

Most browsing APIs take the cursor as an argument. Let’s first see how keys are constructed and then see how to browse the data.

How Key Works

Drip#write returns a key that corresponds to the element you stored. Keys are incremented integers, and the newly created key is always bigger than the older ones. Here’s the current implementation of generating a key:

	​​def time_to_key(time)​​
	​​ time.tv_sec * 1000000 + time.tv_usec​​
	​​end​​

Keys are integers generated from a timestamp. In a 64-bit machine, a key will be a Fixnum. The smallest unit of the key depends on μsec (microsecond), so it will collide if more than one element tries to run within the same μsec. When this happens, the new key will increment from the latest key by one.

	​​# "last" is the last (and the largest) key​​
	​​key = [time_to_key(at), last + 1].max​​

Zero becomes the oldest key. Specify this number as a key when you want to retrieve the oldest element.

Browsing the Timeline

So far, we’ve tried the read, read_tag, and head methods for browsing. There are other APIs:

read, read_tag, newer

Browsing to the future

head, older

Browsing to the past

In Drip, you can travel the timeline forward and backward using these APIs. You can even skip elements by using tags wisely.

In this section, you’ll find out how to seek for certain elements using tags and then browse in order.

The following pseudocode takes out four elements at a time. k is the cursor. You can browse elements in order by repeatedly passing the key of the last element to the cursor.

	​​while true​​
	​​ ary = drip.read(k, 4, 1)​​
	​​ ...​​
	​​ k = ary[-1][0]​​
	​​end​​

To emulate the preceding code, we’ll manually replicate the operation using irb. Is your MyDrip up and running? We also use MyDrip for this experiment. Let’s write some test data into Drip.

	​​# terminal 1​​
	​​% irb -r my_drip --prompt simple​​
	​​>> MyDrip.write('sentinel', 'test1')​​
	​​=> 1313573767321912​​
	​​>> MyDrip.write(:orange, 'test1=orange')​​
	​​=> 1313573806023712​​
	​​>> MyDrip.write(:orange, 'test1=orange')​​
	​​=> 1313573808504784​​
	​​>> MyDrip.write(:blue, 'test1=blue')​​
	​​=> 1313573823137557​​
	​​>> MyDrip.write(:green, 'test1=green')​​
	​​=> 1313573835145049​​
	​​>> MyDrip.write(:orange, 'test1=orange')​​
	​​=> 1313573840760815​​
	​​>> MyDrip.write(:orange, 'test1=orange')​​
	​​=> 1313573842988144​​
	​​>> MyDrip.write(:green, 'test1=green')​​
	​​=> 1313573844392779​​

The first element acts as an anchor to mark the time we started this experiment. Then, we wrote objects in the order of orange, orange, blue, green, orange, orange, and green. We added tags that corresponded with each color.

	​​# terminal 2​​
	​​% irb -r my_drip --prompt simple​​
	​​>> k, = MyDrip.head(1, 'test1')[0]​​
	​​=> [1313573767321912, "sentinel", "test1"]​​
	​​>> k​​
	​​=> 1313573767321912​​

We first got a key of the anchor element with the “test1” tag. This is the starting point of this experiment. It’s a good idea to fetch this element with the fetch method.

Then, we read four elements after the anchor.

	​​>> ary = MyDrip.read(k, 4)​​
	​​=> [[1313573806023712, :orange, "test1=orange"],​​
	​​[1313573808504784, :orange, "test1=orange"],​​
	​​[1313573823137557, :blue, "test1=blue"],​​
	​​[1313573835145049, :green, "test1=green"]]​​

Were you able to read as expected? Let’s update the cursor and read the next four elements.

	​​>> k = ary[-1][0]​​
	​​=> 1313573835145049​​
	​​>> ary = MyDrip.read(k, 4)​​
	​​=> [[1313573840760815, :orange, "test1=orange"],​​
	​​[1313573842988144, :orange, "test1=orange"],​​
	​​[1313573844392779, :green, "test1=green"]]​​

It should return the next three elements. This is because there are only three elements newer than the cursor k. What will happen if you try to read further? It should block the read as you expect.

	​​>> k = ary[-1][0]​​
	​​=> 1313573844392779​​
	​​>> ary = MyDrip.read(k, 4)​​

Let’s write from another terminal and check whether the read gets unblocked.

	​​# terminal 1​​
	​​>> MyDrip.write('hello')​​
	​​=> 1313574622814421​​

terminal 2 should be unblocked. Next, let’s filter using read_tag. Let’s rewind the cursor and try it again.

	​​# terminal 2​​
	​​>> k, = MyDrip.head(1, 'test1')[0]​​
	​​=> [1313573767321912, "sentinel", "test1"]​​

We’ll request to read two to four elements with the “test1=orange” tag.

	​​>> ary = MyDrip.read_tag(k, 'test1=orange', 4, 2)​​
	​​=> [[1313573806023712, :orange, "test1=orange"],​​
	​​[1313573808504784, :orange, "test1=orange"],​​
	​​[1313573840760815, :orange, "test1=orange"],​​
	​​[1313573842988144, :orange, "test1=orange"]]​​

We get four oranges. Let’s update the cursor and do the same operation again.

	​​>> k = ary[-1][0]​​
	​​=> 1313573842988144​​
	​​>> ary = MyDrip.read_tag(k, 'test1=orange', 4, 2)​​

This blocks your terminal because there are no orange elements newer than the cursor. Let’s write two oranges from another terminal, and read_tag should start working.

	​​# terminal 1​​
	​​>> MyDrip.write('more orange', 'test1=orange')​​
	​​=> 1313575076451864​​
	​​>> MyDrip.write('more orange', 'test1=orange')​​
	​​=> 1313575077963911​​

	​​# terminal 2​​
	​​>> ary = MyDrip.read_tag(k, 'test1=orange', 4, 2)​​
	​​=> [[1313575076451864, "more orange", "test1=orange"],​​
	​​[1313575077963911, "more orange", "test1=orange"]]​​

You’ve just learned how to seek and read using tags, as well as filter using read_tag. These are basic idioms to browse through Drip.

There are also other utility methods.

​​Drip#newer(key, tag=nil)​​

This will return one element newer than the key. You can also specify a tag. newer is a wrapper method of read and read_tag. If there are no matching elements, it will return nil rather than block.

​​Drip#older(key, tag=nil)​​

This will return one element older than the key. You can also specify a tag. If there are no matching elements, it will return nil rather than block.

Oh, I almost forgot. There is an API to retrieve a value when you know the exact key.

​​Drip#[](key)​​

	​​>> k, = MyDrip.head(1, 'test1')[0]​​
	​​=> [1313573767321912, "sentinel", "test1"]​​
	​​>> MyDrip[k]​​
	​​=> ["sentinel", "test1"]​​

This returns an array of a value-tag pair. It won’t return a key.

Those are all the major APIs. By the way, you haven’t seen the each method often used in Ruby, have you? I’ll explain the reason why in the next section.