Sunday, October 27, 2013

FlightGear Rembrandt: HAF -> SFO

The windy walk back from work during dusk reminded me that I do in fact live in San Francisco. And although my work schedule doesn't allow me to get out of town all that much, fortunately my flight simulator tries to fill in the gap. A couple days ago I decided to flip on the new "Rembrandt" setting in the flight sim, FlightGear, and I found myself sinking into an entirely new cockpit, as if it were the real thing. As I took a short hop tonight from Half Moon Bay Airport to San Francisco International, the airport lights flickered just beyond the cabin glow of my cockpit. And just to make things feel truly real, immaculately surreal, the sim's real-time weather brought me from the fog of HAF airport to a crosswind landing at SFO.



A couple weeks ago, I did actually get to venture inside a cockpit at SFO, this one a 737-900. My brother's friend from church gave us tickets to a United Airlines show, where I got to take a snapshot of my niece in her future profession (or so we hope!).


To my amazement, as soon as I took these snapshots from my Android phone, Google identified them as coming from a 737. It even guessed a 737-800--pretty close! Now if FlightGear could get its planes identified by Google as "the real thing," that would be rather impressive. And judging by the rendering in Rembrandt, I think that day may not be so far off.


In any case, as much as I enjoy the relaxing realism of a virtual flight after a long day, I'm thankful for the real pilots keep our real bodies safe in the air, day after day after day.


Monday, October 14, 2013

Installing Matplotlib and Mayavi on Mac

I had lots of fun with Matlab in grad school, but I always had in the back of my mind that at some point my educational subscription would run out. Thus started my slow search for alternatives, preferably open source alternatives with combined graphing capabilities.

I also had lots of fun with Python, so my hunt brought me to the Matplotlib, a Pythonic scientific graphing tool. While Matlab has a single application (plus many toolkits) to install, the Pythonic solution has no less than Python + Numby + Scipy + Matplotlib to get things rolling. Installing Matlab is as simple as starting and installer and letting it do the work, whereas installing Matplotlib was another story. Allow me to document that story here in the likely chance that I'll otherwise forget it.

Matplotlib can be installed through a single solution provided by a company named Enthought, but my hope was to install everything using standard open-source installers. Here's my setup and the specific requirements I was aiming for:
  • Mac OS 10.8
  • Homebrew for package management
  • Python installed from Homebrew rather than from Apple XCode
  • Virtualenv to isolate a "scientific Python" environment
  • Matplotlib as a primary graphing tool
  • Mayavi as an alternative 3D graphing tool
Installing Homebrew

Back in college I became an avid Linux user, and part of both the joy (and occasional angst) of Linux was the package management system. rpm, yum, and apt-get all provided ways to keep software up to date without having to check individual sites for updates.

When I had to use Windows more often, I found the Cygwin system a boon both for accessing all my Linux/bash scripts on Windows as well as keeping everything up to date. At some point, I did install Matplotlib via Cygwin but, alas, did not document instructions and have since forgotten the details of the process.

Now that I'm using a Mac, I found Homebrew to meet all my package management needs. Installing it was a matter of running a simple command (scroll to the bottom of that page). Keeping it up to date involved brew update; brew upgrade, and that was that.

Installing Homebrew-based Python

One of the tricky aspects of Homebrew is that Python can come from multiple locations. Apple XCode already includes Python, so by default Python will run, albeit an older version. Installing the Homebrew-based version can be a help as it brings a newer version and also allows customization so that it will work with a wider variety of packages. The Homebrew-based package also apparently brings pip with it for additional installations.

To install Python 2.x, I used the command:

brew install python --with-brewed-openssl
Python 3.x could also apparently be installed side-by-side with Python 2.x by replacing "python" with "python3", although I haven't yet tried out 3.x.

I did come across some errors, most likely based on a prior installation of pip outside of Homebrew. This SO answer and blog post helped me revive the installation.

Installing pip and virtualenv to install more packages

Now that we've used one package manager, we get to use another package manager, this time pip, a Python package manager. And the first package to install is a tool to manage multiple sets of packages. Talk about recursive package management, huh?

Following this guide, my first step was to install virtualenv, which allowed me to isolate the subsequent packages in case I later wanted to install a different, potentially conflicting set of packages. A side benefit is that if I made a mistake during installation, it was a matter of deleting a virtual environment rather than trying to clean up my entire system.

I used this command to install virtualenv, using pip provided by the Homebrew-based Python.

pip install virtualenv
To keep my virtual environments together, I made a separate directory and then created a new environment, which is started through the activate script.

mkdir ~/virtualenvs
cd ~/virtualenvs
virtualenv pystat
source pystat/bin/activate

Installing Numpy + Scipy + Matplotlib

And now, time to install what we set out for! First, be sure to deactivate the virtualenv session to install from within a normal shell session prior to running further brew commands.
UPDATE (9/5/2016): Looking back through this post, I think that the pip commands still need to be within the virtualenv to make isolated environment relevant. Also, I'm not sure how isolated this entire configuration is since the Homebrew installations occur outside of the environment, and updates appear to break the installation. For example, a recent incompatibility with vtk appears to have rendered my installation unusable at least for now.
deactivate
pip install numpy
brew install gfortran
pip install scipy
brew install freetype
pip install matplotlib

Here's a screenshot of visualizing football models with Matplotlib.


Installing Mayavi

While testing out Matplotlib for 3D histograms, I came across a few rendering issues and this faq from the Matplotlib site itself that suggested using Mayavi as a more powerful 3D engine. Installing Mayavi was no laughing matter, but it did work in the end, particularly with the help of this guide.

Installation involving tapping another Homebrew repository, homebrew/science,

brew install cmake
brew install qt
brew install pyqt
brew install --python --qt vtk
brew install wxmac
pip install configobj
pip install envisage
pip install mayavi

During the final step, I was initially unable to install mayavi because of an error in finding the vtk package, despite having successfully installed it. Apparently vtk gets installed in the Python system site package and is not accessible from within the virtualenv. My workaround was simply to turn off the switch that was preventing access to these packages while in a virtualenv:

rm ~/virtualenvs/pystat/lib/python2.7/global-site-packages.txt
pip install mayavi

Mayavi is installed!

UPDATE (2/24/2014): While trying to update Homebrew today, I ran into a few errors. The first error stated:
vtk: Unsatisfied dependency: matplotlib
Homebrew does not provide Python dependencies; install with:
  pip install matplotlib

Matplotlib was installed within the virtualenv and does not appear to be visible to Homebrew. I ended up having to activate the virtualenv where matplotlib was installed and running the Homebrew update from there. vtk did show up in the Homebrew global Python site-packages directory (/usr/local/lib/python-2.7/site-packages), even when installed from the virtualenv.

I also encountered a error on updating wxmac:
/usr/local/Cellar/wxmac/3.0.0.0/include/wx-3.0/wx/strvararg.h:25:14: fatal error: 'type_traits' file not found
    #include <type_traits>
             ^
1 error generated.
error: command 'clang' failed with exit status 1

Apparently this error has been reported and commented upon extensively several months back with a workaround. The solution was to run:
brew edit wxmac
and add the line:
ENV.append_to_cflags "-stdlib=libc++"
to the 'cd "wxPython" do' block (currently line 39, where the line can be added after the existing append_to_cflags call).

But alas, it would not run. I received the error message:

This program needs access to the screen.
Please run with a Framework build of python, and only when you are
logged in on the main display of your Mac.

which fortunately had this handy SO solution with a script to run Python from within the requisite environment (or see this alternative script). Here's a screenshot of visualizing that same football model but with Mayavi.


Now I'm enjoying both Matplotlib and Mayavi home-brewed in a cozy virtual environment. With a real coffee in hand, of course.