Wednesday 27 February 2013

How to create beautiful reports in Ruby, the easy way

The world as you know it
Think about business software: Reporting is an essential part of it. It's the lifeblood of usefulness to the enterprise, it's an integral part of an application's value and many applications are really data presentation engines, such is the importance of reporting and the ability to generate reports. 

From invoices to understandable graphical presentations of  complex data sets, reporting is a serious topic, covering a wide range of skills and sciences, from high end mathematics to the art of designing something that is beautifully presented yet functional and useful.

Ruby has phenomenal data extract, processing and rapid development capabilities. Ruby on Rails is best of breed rapid, robust and maintainable web application development.

Reporting may be an essential part of business, but despite all their suitability to the enterprise and business world, Ruby and Rails have been dogged by a big black hole when it comes to reporting. Sure you may produce lines of text output quickly, but creating beautifully crafted reports to compete with the best out there has been a very un-Ruby, un-Rails experience: Messy, buggy, slow to produce, a nightmare to maintain especially when it came to reworking presentation.

Well, it was like that.

A problem solved
You are about to discover the solution that you have been waiting for. 
You will be able to design beautiful reports quickly and easily and populate them with data from your Ruby or Rails system  You can publish these reports in any format you want including the highly recommended PDF format. You see how to email them easily.

Most importantly, in many businesses such as digital agencies, the person who designs the user-facing parts of the system such as the reports is not the developer. You are about to discover the secret that will let a designer design the report to their heart's content and you will be able to simply drop the new design into your report without so much as a line of code changing.

Secret ingredient for the magic
To create great looking reports, you need a world-class design tool. Something familiar and non-technical to designers. Something that gives them quick editing, precise layout capabilities, many built in tools and enables external tools all in a simple, easy to use graphical interface. Meet LibreOffice, the fork of the OpenSource phenomenon, OpenOffice.

LibreOffice is our secret weapon because it does what not even OpenOffice can do: It can run simply and easily under a program's control. No other office suite other than OpenOffice and LibreOffice can run on the wide range of operating systems out there, including the ubiquitous Linux. Enterprise servers run Unix, Linux, BSD. Apple's OSX is BSD-based. Yes, even Windows is supported.

The secret codeword for success
The keyword that unlocks beautiful reports is: headless.

The term headless means that LibreOffice/OpenOffice run without the graphical user interface. In headless mode, they are a utility on a server, they can be used in batch processing, by other software such as the code that you are writing. You can deploy them onto a web server and use them to generate beautiful documents, crunch numbers in a spreadsheet and convert between file formats.

However, until recently, there was a catch.

Before the brilliant folk at the LibreOffice project changed things, OpenOffice is/was capable of running headless, but as a server with a non-trivial, poorly documented API to drive it. This meant that to use, your best choice was using convoluted java interfaces. A python command line tool is available too.

This was messy. 

OpenOffice interfaces came with multi-threading limitations. The system configuration overheads for setting up headless OpenOffice mean that in many organisations using this solution would not be an easy roll-out. Such server-level configuration often involves multi-team agreement and engagement i.e. the difference, in reality, between can do and can't do.

The big break-through with LibreOffice headless mode is that it enables you to simply call LibreOffice on the command line directly as a command line tool.

The format of this call (to generate a PDF) is:
   libreoffice --headless --invisible --convert-to pdf

In Ruby, you cam simply back-quote (`) that command to run the conversion.

Completing the solution
In the next post we will look at how a document designed in LibreOffice can become a template for a report. The technology that we use is odf-report but beware, there are traps and tricks to know when using this bridge.

Bullet point summary
  • Design reports using LibreOffice
  •  Use odf-report to populate the report in your Ruby or Ruby on Rails application
  • Produce the completed report from your Ruby code by calling LibreOffice in headless mode to convert the LibreOffice file to a PDF (or whatever document type you wish).

Ruby/Gem upgrade problems -> 1.8 to 1.9 and solutions

If, following a RedHat linux (EL5) update, your ruby is broken with a message such as this:

/usr/bin/ruby: symbol lookup error: /usr/local/lib/ruby/gems/1.9.1/gems/json-1.7.7/lib/json/ext/ undefined symbol: rb_intern2

have a look at your ruby path. RedHat puts the updated ruby executable in /usr/local/bin/ruby. You will probably need to do this:

# mv /usr/bin/ruby /usr/bin/ruby_old
# ln -s /usr/local/bin/ruby /usr/bin/ruby


Another problem that is commonly encountered is with the gem command after an update:

You get an error dump like this:
# gem list
/usr/local/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb:82:in `rescue in rescue in ': uninitialized constant Gem::ConfigFile::RbConfig (NameError)
        from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb:64:in `rescue in '
        from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb:60:in `'
        from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb:36:in `'
        from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/gem_runner.rb:9:in `require'
        from /usr/local/lib/ruby/site_ruby/1.9.1/rubygems/gem_runner.rb:9:in `'
        from /usr/local/bin/gem:9:in `require'
        from /usr/local/bin/gem:9:in `

The key here is the bit that says "Gem::ConfigFile::RbConfig (NameError)" 

What this is telling you is that it cannot find the RbConfig class. This is a bug in the upgrade from 1.8 to 1.9. There is a line that needs to go into your gem command. Add this to your gem file:

require "rbconfig"

You can locate the gem command on Linux using this command:

which gem

Then simply vi the file and add the above line in.