I was looking for a way to make a search on Google and getting the results back on a one-by-one basis instead of getting them by 10 per query to the api.
In Python I’d use generators for this and I was looking for something similar in Node. As far as I could figure out actual generators are only available starting from v11.x and thus not yet in 10.28, the current stable version. And even if they were, they wouldn’t be in CoffeeScript yet.
The closest solution that I found was Streams. It was a bit tricky figuring these out and getting them to work in CoffeeScript, hence this post.
Streams in node by default handle with Buffer values. (Which can also be seen as strings since calling
.toString() on one will convert it to a string.)
Now when we run this:
The console output will be:
- CharStream takes a string and it’s output is each character in that string one by one
- UpperCaseStream then takes the results from step 1 and transforms them to uppercase characters.
- Finally LogStream takes those uppercased values and logs them in the console
Note that in
_read at the end null is pushed, this is to tell the consumer of the stream that the stream has ended.
If you don’t do this the consumer will call
_read again, causing the stream to effectively never end. The output will then be an infinite sequence of
['a', 'b', ' ', 'c', 'a', 'b', ...]
Pushing things will (might?) trigger _read to be called again.
objectMode = true.
It’s important to do this in the call to
super and not just setting it on
Running the following will cause the values of the name properties to be logged in uppercase.
Google search results as a stream
What I eventually ended up with for getting the search results from Google one-by-one is the following:
Note that the Google web search api is deprecated. This is just some example code demonstrating what I could figure out from streams. I’m sure that there is a cleaner way of doing something like this and would love tips and improvements in the comments.