30 September 2013

A Sliver of Ruby #2: Ruby's Logical Operators (and their proper use)

Ruby has two triads of logical operators. For those new to Ruby, this can be a source of confusion: in some common cases, the corresponding operators from each triad are interchangeable; in other cases, their behavior is quite different.

The first set of the triad is && , || and ! — the "logical and, logical or, logical not."  This group is best used within an expression, to combine a bunch of conditions together.

The second is and , or and not— the "logical composition operators".  These operators are intended to be used between statements to conditionally combine them.

The Same on the Surface But Different Underneath

Both set of operators calculate expressions the same way.

That is: both && and and take two operands and logically and them together; both || and or take two operands and logically or them together.  Likewise, ! and not both negate the expression.

apple = true
orange = false

apple && orange  # => false
apple and orange # => false
apple || orange  # => true
apple or orange  # => true

!apple           # => false
not apple        # => false
!orange          # => true
not orange       # => true

This is where the similarities end. There are two fundamental differences.

Difference #1: Short-Circuiting

&& and || can short-circuit the remainder of the expression; whereas and and or only short-circuit between the two operands.

# Only if the souschef fails to wash, will the chef wash.
# but in either case, the chef always slices the apple.
souschef.wash(apple) or chef.wash(apple) and chef.slice(apple)

# if the souschef does peel the orange, nothing more happens.
# else, the chef peels the orange (and if successful) slices it.
souschef.peel(orange) || chef.peel(orange) && chef.slice(orange)

Difference #2: Precedence

and, or and not are among the lowest precedence operators in the Ruby language.  In contrast, &&, || and ! are relatively higher.  This is particularly important when your expression includes other operators that sit between these two sets in the precedence ranking.

Here's the complete set of operators in Ruby, in order of precedence.

Symbol
Name
::
Constant resolution operator
[] []=
Element reference operator and element set operator 
**
Exponent operator (raise to the power of...) 
! ~ + -
Unary operators: not, complement, unary plus, unary minus 
* / %
Multiplicative operators: multiply, divide, modulo 
+ -
Addition and subtraction operators 
<< >>
Bitwise shift operators (left and right) 
&
Bitwise "and" operator
| ^
Bitwise "or" and bitwise "xor" operators
> < >= <=
Comparison operators 
== === <=> =~
Equals, same object, "spaceship"/compare, and regex match operators
&&
Logical "and" operator
||
Logical "or" operator
.. ...
Range operators: inclusive and exclusive
? :
Ternary operator 
=
Assignment operator
defined?
Symbol existence operator 
not
Logical negation operator
or and
Logical composition operators
if unless while until
Flow control operators 
begin/end
Block expression 
Table 1 —  Ruby operators in order of precedence.  Highlighted sit between the two triads.

For example:

Using the Logical "and" operator, this statement is non-sensical...

# The chef washes (nil), and food becomes
# whether she is successful AND whether orange is truthy
food = orange && chef.wash(food)

whereas, with the logical composition "and", we have a series of statements strung together:

# The food is orange
# And the chef washes the food (i.e. the orange)

food = orange and chef.wash(food)

That's a lot of detail to keep in mind. The point is to appreciate that even though the two operator sets appear to behave the same superficially, deep down, they are different and not always interchangeable.

Conclusion

Where you go from here depends on the skill level of the "average committing developer" for your product, both present and future (see Dryfus Model of Skill Acquisition for definitions).

Novice and Advanced Beginner

In this case, you're likely working in a shop that leans more toward being productive over engineering a solution.

The most professional move here is to keep it as simple as possible.  In this way, the system has a better chance of being maintainable.

Avoid and prohibit use of the logical composition operators (and, or, and not) all together.  There is no significant loss of expressiveness (i.e. there are no significant statements that you suddenly can't write) and it avoids this confusion completely.

Competent 

There is room/expectation for some deliberation, but you're mostly writing application code and not so many frameworks and/or libraries.  That is: your code is rather concrete and therefore more straightforward.

In these circumstances, you can certainly make use of the additional expressiveness that the logical composition operators bring.  However, you'll likely want to define the context for when programmers ought to use each.  Include include these in your coding style guide.

Here's a pair as an example...

Guideline #1: Use &&, || and ! for boolean expressions

Focus/limit your use of the symbolic set of logical operators (&&, || and !) for composing boolean expressions.

For example:
if params[:order] && params[:order][:coupon_code]

not

if params[:order] and params[:order][:coupon_code]
... and ...

return false if !Rails.env.production? ||
                !Spree::Config[:check_for_spree_alerts]
instead of
return false if !Rails.env.production? or
                !Spree::Config[:check_for_spree_alerts]


Guideline #2: Use and, or, and not for flow control

Use the word set (and, or and not) for conditionally stringing statements together. In other words, for flow control.

Examples:

respond_with(@order) { render :edit } and return
not
respond_with(@order) { render :edit } && return

... and ...

get_from_cache(id) or load_from_disk(id) and send_email()
instead of
(get_from_cache(id) || load_from_disk(id)) && send_email()

Proficient and Expert

You're in a the-product-is-the-software company and building not just the end products, but an ecosystem of compose-able components, flexible frameworks and multi-purpose libraries.

Chances are, you're not reading this article. :)  But for completeness' sake, the advice here is to use the full power of the toolset you have.

Avoid unnecessary parenthesis and loss of expressiveness by using the right operator for the job.

Example: Pre-Checks


(from https://github.com/spree/spree/blob/master/core/lib/spree/responder.rb):

def to_html
  super and return if !(on_success || on_failure)
  has_errors?
    ? controller.instance_exec(&on_failure)
    : controller.instance_exec(&on_success)
end

Here, there's an early return at the top of the method if there are no hooks registered.  Pre-checks like this are reasonably easy to learn idioms and help avoid verbose guards (e.g. an if statement surrounding the contents of the method).

Notice, BDQ's thoughtful use of both kinds of logic operators that visually convey the difference uses (i.e. he used and specifically for logical composition and || for boolean math).

Example: Precedence

(from https://github.com/rails/rails/blob/master/activerecord/lib/active_record/associations/has_many_association.rb)

def count_records
  ...
  @target ||= [] and loaded! if count == 0
  ...
end

Here, Leighton is making use of the fact that assignment has a higher precedence than the logical composition operator.  This is a little more straight-forward and esthetically pleasing than:

def count_records
  ...
  (@target ||= []) && loaded!() if count == 0
  ...
end

(I added the second set of parens after "loaded!" to call out the fact that they would be needed had the method required parameters.)

References

27 September 2013

A Sliver of Ruby #1: Truthiness

In Ruby, there are only two values that mean "false": the literal false and nil.

Everything else is truthy; in order words, evaluates to true in boolean expressions.

This can be confusing for programmers who have experience with other languages where 0 (zero) is falsey.  Not so in Ruby.

For the excruciating details, check out this gist: https://gist.github.com/jfarmer/2647362

25 September 2013

git-grep: Faster at Finding Socks Than Your Mom (and better looking)

This one is a meticulous 30 minute investment that pays off in spades the first couple of uses.

Imagine you're in one of these situations:
  • you are entering into a new codebase and aren't familiar enough with it to navigate around with ease — more of a challenge when you are using dynamically typed languages like Ruby or JavaScript where and IDE is less able to properly parse for and locate definitions of used classes/objects/methods (think of, for example, how Open Classes means that method definitions can come from, technically, anywhere). In these cases, having the ability to quickly text-search for other uses and definition is a major time-saver.
  • you're trying to make heads-or-tails of an unhelpful error message — looking for parts of that message or related concepts might give a clue as to what code is at play, at least when the error is being reported.

find | xargs grep

For years, my toolset of use, in these cases has been find | xargs grep (and for those who want that Unix feeling on Windows, check out cygwin or the like).  The syntax is relatively straight-forward:

$ find . -print0 | xargs -0 grep 'def abstract!'
Here,
  1. The "find" tool is looking for all of the files at the current working directory (i.e. ".") and all sub-directories and spewing the pathname of each to Standard Out.  The "-print0" tells find to separate each path with a null character (ASCII zero);
  2. That stream is getting piped to "xargs", which given the "-0" (dash zero) parameter tells it that parameters are null-separated (this and the "-print0" of find are working in concert, here);
  3. xargs is repeatedly invoking "grep" with as many filenames as grep can handle (but no more) in a bunch;
  4. "grep" takes each pathname, opens the file and searches for the supplied text, (here, 'def abstract!') [note: the single quotes are required here so that the shell doesn't attempt to interpret any special characters in the string, the exclamation point being one].
This is great.  Like I said, I've been using this little pipeline for a very long time.  But I have discovered for myself an even better tool for the job: git-grep.

git-grep

git-grep is an incredibly useful source code searching tool.  It's built into git, so if you've already taken the red pill, this is just more drunken boxing-fu for free.

... faster than find | xargs grep ...

In essence, git-grep does a fast search not over a set of files, but its own speedy index of file contents. So, without talking about feature sets, git-grep is much faster than find | xargs grep.  Here's a real-world example: a download of the HEAD of rails' master branch (5,720 files, 242,229 lines, 9,578,293 characters):
$ time find . -print0 | xargs -0 grep 'def abstract!'
./actionpack/lib/abstract_controller/base.rb: def abstract!
real 0m2.072s
user 0m0.079s
sys 0m0.477s
vs.
$ time git grep 'def abstract!'
actionpack/lib/abstract_controller/base.rb:33: def abstract! 
real 0m0.070s
user 0m0.035s
sys 0m0.081s
that's 2 seconds vs. 70 milliseconds.  So, faster.

... with appropriately configurable output ...

git-grep comes with a couple of features that grep simply doesn't have:
  • extensive color control — where grep allows you to specify the color for matches, alone, git-grep allows you to configure a number of elements of the output making it far easier to visually parse.
  • identify the enclosing "function" — git-grep can be configured to not just present surrounding context (i.e. X number of lines before and after your match to help provide "context" to the match), but also locate the "function" in which the line belongs. More details in a subsequent post, but there's built-in support for Ruby, Java, C#, Objective-C, PHP, Python, and a bunch of others.  (for more information, look into Git attributes.  Here's the rabbit hole.)
Here's a side-by-side visual comparison:

find | xargs grep output.  This is fine and dandy.

... which is certainly not bad: the matched line is distinguished from context and you have line numbers.  Compare that with this:

git-grep output.  I find this easier to read and like the DRY output with "function" identified

... where the filename is displayed just once at the top of the match (configurable).  and we get the added bonus of identifying the enclosing module (underlined), this is what matched as a "function" in this case.

In the second example, I've got a bash alias defined that invokes a git alias:
~/.bashrc (excerpt)
alias gg='git gp'
and the git alias contains the set of parameters I typically want:
~/.gitconfig (excerpt)
gp = grep -I --heading --before-context 2 --after-context 2 --show-function --untracked --extended-regexp
(my full configuration is in the links at the end of the article).

... one caveat ...

There's really only one "caveat" in using git-grep: you need to be within a git repo.  If you attempt to do a git-grep outside of a repository, you'll get a reasonable error message:
fatal: Not a git repository (or any of the parent directories): .git
All that's required to remedy this error message is to initialize a git repo:
$ git init
And it even though none of the files are in the git repository let alone the index, git-grep is still faster...
john@slick:development [649]$ time gg "some very unique string"
projects/jtigger/ruby/rails-master/my-file.txt
1:some very unique string
real 0m32.300s
user 0m2.073s
sys 0m6.628s
john@slick:development [650]$ time find . -print0 | xargs -0 grep "some very unique string"
./projects/jtigger/ruby/rails-master/my-file.txt:some very unique string
real 1m7.528s
user 0m4.291s
sys 0m10.395s

Other Source Searching Tools

Of course, git-grep isn't the only game in town.  Nor is it the fastest.
http://beyondgrep.com/more-tools/

References



21 September 2013

Broken Link? Check The Way Back Machine!

Turns out, there's a Time Machine for the World Wide Web!  It's called "The Way Back Machine" and it's a reasonable chronological archive of crawl-able websites.


microsoft.com in 1998, the earliest archived copy of the site.
Imagine you're using the 'Net to answer some question (random example: "In Ruby, what the heck does class << self; self end do?") and you find this link to what all the surrounding text seems to indicate is an articulate explanation.  With great anticipation, you click!  ...only to find... a co-ed :-/


rubygarden.org on September 21, 2013.  Cute chick, but not what I was looking for...

Something just like this happened to me, this morning.  I was reviewing the Singleton pattern syntax in Ruby in some of my old notes which included a reference to the Ruby Garden.  This was a wiki that David Black, Chad Fowler, Rich Kilmer were maintaining back in the day, until it was overrun with spam.

Alas! While I was not able to find the page I was looking for, a trove of other useful parts of the wiki are safely tucked away in the Way Back Machine archive!


The RubyIdioms page in the rubygarden.org wiki on September 14, 2007

The next time you hit a broken link and it breaks your heart, remember you can rekindle the flame at the Way Back Machine!

19 September 2013

(Most Of The Time) I ♥ Chrome

There I was, just minding my own business, writing a detailed message to a friend and ex-colleague on LinkedIn (about 10 minutes into the message)... when my right hand slid the mouse over and accidentally clicked the close button on that tab in Chrome.

"Nooooooooooooooooooooooo!" I panicked.


It wasn't just the time spent writing it (and would have to spend rewriting it), it was all of the turns of phrase and word selection that just went down the drain.  And then I took a deep breath and remembered that Chrome has this feature where you can open recently closed pages...


I hit ⌘T, clicked on the "Recent Closed" menu on the bottom right-hand corner of the page and selected the "LinkedIn" item.


Back popped up my LinkedIn session, complete with the entire message I had written.  It's in moments like these that I really do appreciate the fine engineering that went into this browser.

18 September 2013

Solarized

In the process of spending way too much time selecting colors for my git-grep setup, I stumbled into one man's study of color for daily use...

http://ethanschoonover.com/solarized

Solarized.  Ethan Schoonover's study of useful color schemes on computers.

"Solarized is a sixteen color palette (eight monotones, eight accent colors) designed for use with terminal and gui applications."



2013-09-21 Update:

Found in the wild: Solarized (both light and dark) schemes used by the Ace Editor.


http://ajaxorg.github.io/ace/build/kitchen-sink.html

17 September 2013

Overloading the constructor in Ruby



Ruby's construction process is flexible: it allows you to short-circuit the object construction process itself, allowing you to provide your own instance of the object being construction.

An example is in order:

https://gist.github.com/jtigger/6595108

This feature is handy, for example, when backing object instances with a cache.  In this case, when a client programmer "constructs" an object the first time, you'll want to build it, as usual and return that instance (see the call to super in the gist).  However, if they request the same object a little later, and it's still in the cache, you'd rather skip creating the object and just return the instance that's in the cache.

Unpacking that construction flow a little deeper...

  • first call to Foo.new(1) results in a "cache miss" and therefore the new() method of the super-class of Foo (i.e. Object.new()) is invoked by calling super.
  • the second call of Foo.new(1), finds the instance of Foo created during the first invokation.  This is a "cache hit" and merely returns that instance from the cache.


Pretty cool!

Thanks to Jörg W Mittag on StackOverflow for this idiom.


p.s. also note that super not only calls the same method on the super-class, but automatically passes the same parameter list that was sent to the current method.  In the example in the gist, id is passed to Object.new().

15 September 2013

How git Saved My Bacon Today...

Okay, so that title is a little dramatic, but it sure felt amazing to use my version control system to pull me out of a hole.

(this is a little gem that I'm resurrecting from a prior blogroll... hence the 2010 dates)

The story so far...
I had been working my way through the excellent text Agile Web Development with Rails (4th edition) by Sam Ruby.

I let myself be led by the nose through a couple of working sessions without running all of the functional tests.  Suddenly, there I was on Iteration F4 and my test complained:


$ rake test
.
.
.


Started
.F....................
Finished in 0.538259 seconds.


  1) Failure:
test_should_destroy_cart(CartsControllerTest) [test/functional/carts_controller_test.rb:43]:
"Cart.count" didn't change by -1.
<1> expected but was
<2>.

However, this particular test failure wasn't described in the text (oh noes!).  I was on my own to troubleshoot the underlying issue.  I tried a number of typical first-line troubleshooting techniques: sprinkling a few pretty-prints here and there to get a sense for why the destroy wasn't actually destroying.  But to no avail.  And since I hadn't run my functional tests in some time, I didn't know when I introduced the defect. :-/ 

git bisect to the rescue!
Luckily, I had been diligent at least about committing my work to a local git repo.  I was committing each "iteration" (unit of work from the text), so I had a reasonably high-fidelity history.

With git bisect, I was able to quickly hunt down the offending commit.

First I went back in time a bunch of commits (about 8 commits back) and ran the rake test.  They succeeded!  So, at least I know the tests were running back then and branched there as "last_known_good":

(master) $ git branch -f last_known_good 4586dadff6cf838262c48149ca265e5226cb80b1
(master) $ git co last_known_good
(master) $ rake test
Started
...............
Finished in 0.399094 seconds.

15 tests, 24 assertions, 0 failures, 0 errors, 0 skips

I then initialized a bisect session:

(master) $ git bisect start
(master) $ git bisect bad
(master) $ git bisect good last_known_good
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[8aab0db8f45e8a0c2f5ffc10e6f2382d90c249e8] (E1,E2) Added 'quantity' to Line Items

This first step is the halfway point between the "last_known_good" branch and my master branch (hence the name: git is bisecting the history, doing a binary search until the actual last known good commit (or even better, the first known bad commit) is identified.

Now, it's just a matter of running my test and letting bisect know whether they succeeded or not.  bisect took care of doing the binary search through the history to find the actual last known good:

jonamac:work ((no branch))$ rake test
Started
......................
Finished in 0.541449 seconds.

22 tests, 34 assertions, 0 failures, 0 errors, 0 skips

that one was good.  So, I just tell git bisect that.  It then finds the midpoint commit between this commit (about the 4th from the top)

$ git bisect good
Bisecting: 1 revision left to test after this (roughly 1 step)
[a17b740253ceb4e4df5f055808aa638bee269a7f] (F1) Added partials for Line Item and Cart.

git tells me that he's moved to that midpoint.  I just run my rake test again and tell git about the results.

$ rake test
Started
.F.E...F.EEEE...EEEE..
Finished in 0.557046 seconds.

  1) Failure:
test_should_destroy_cart(CartsControllerTest) [test/functional/carts_controller_test.rb:43]:
"Cart.count" didn't change by -1.
<1> expected but was
<2>.
...
22 tests, 22 assertions, 2 failures, 9 errors, 0 skips


oiy!  this one is bad!  Again, I just tell git that fact and it continues to hunt down the source...

$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[bc5470e9380a80448fd61e780d24fe8e1927dec3] (E3) Improved cart display.

...run my test again...

$ rake test
Started
.F....................
Finished in 0.538259 seconds.

  1) Failure:
test_should_destroy_cart(CartsControllerTest) [test/functional/carts_controller_test.rb:43]:
"Cart.count" didn't change by -1.
<1> expected but was
<2>.

22 tests, 33 assertions, 1 failures, 0 errors, 0 skips

and as I tell git about this last one (that I see the failure), it reports that it has found the first bad commit:

$ git bisect bad
bc5470e9380a80448fd61e780d24fe8e1927dec3 is the first bad commit
commit bc5470e9380a80448fd61e780d24fe8e1927dec3
Author: John S. Ryan
Date:   Sun Dec 12 18:00:54 2010 -0800

    (E3) Improved cart display.
    
    + Added ability to empty cart.

There we go!  Something in this commit is the root cause of my issue.  gitk can be helpful to visualize what happened:

gitk visualization of a bisection; it's a hunt for when a particular condition first occurs.

It was bisect, not I who applied those tags that start with "bisect/good" and "bisect/bad".  This is handy to get a sense for what just happened.

Now I've got my smoking gun.

What's left is to simply check out at the actual last known good and apply the diff from the next commit, one logical chunk at a time.  Turns out, the bug was a typo (I wrote "card" instead of "cart").

Post Mortem
There was a deep cause to my problem: I had not run my test suite before I committed (repeatedly) and thus injected a defect that was there for some time.  Had I stuck to my guns and continued to run my tests, I would have found this bug much faster.

That said, it's nice to have a safety net.  Lucky for me, I was diligent about my use of my DVCS.  With git's bisect, I was able to track down the source of the defect pretty quick in a rather reliable process.


RSpec: The Best Example of Living Documentation I've Seen


Here's a beautiful example of Cucumber-based Living Documentation.  Of all things, it's for RSpec, a Ruby testing tool.


https://www.relishapp.com/rspec/


In his book, Specification By Example, Gojko Adzic identifies one of the big benefits  (but not the most important1) of systems like Cucumber is that they can be used to produce Living Documentation.

There's a tool for generating HTML from Cucumber .feature files; it's called RelishApp.


Like anything, it's really about the attitude and aptitude applied to a technique and not the motions themselves that produces a quality result.






1 Adzic makes clear that the most important benefit of SBE is the shared clarity that comes from the collaborative specifying work, itself: that when a subject matter expert, a systems designer and test system designer all work together on defining the spec, the quality of the specification is much greater than what results when these factors are considered in a more serial fashion.

14 September 2013

Rebuilding Rails

Last week I began my re-entry into the world of Ruby and Rails... and loving it! 

Ruby is such a delight to work in. I'm repeatedly composing interesting (novel-to-me) combinations of syntax that just run the first time I try them (or with some obvious 8-bit edits, the second time).


My curriculum:


  • Rebuilding Rails by Noah Gibbs — this well-crafted eBook is a step-by-step guide in building Ruby on Rails from scratch. It's like learning how cars work by building one. Noah has paid particular attention to detail and skillfully guides you through building-up the framework, piece-by-piece.
  • LA Ruby Study Group lead by Cynthia Kiser — I found a local study group. In fact, I found out about the Rebuilding Rails book through this group.
  • Ruby Koans by Jim Weirich and Joe O'Brien — A fantastic way of learning the Ruby by doing Ruby, tiny bits at a time. Ruby Koans is a sequence of unit tests (a path to enlightenment). In the beginning, all of the tests are failing (your karma is broken). Step-by-step you fix each test, picking up one little language concept at a time.
  • New Questions on stackoverflow.com — of course, stackoverflow is a great resource that often floats to the top of Google searches containing programming terms. However, what I'm talking about is watching the New Questions About Ruby feed. If there's a question I might be able to answer, I give it a go. I've found that teaching others is a great way to learn.