Mathematica vs Matlab/Octave

Matmaticalab

After 6 years using Mathematica, 7 years using Matlab, and 3 years using Octave on a relatively frequent basis for classes and academic research, I felt qualified to make known my personal opinion on their relative strengths and weaknesses — at least as they pertain to the types of things that I’ve wanted to do over the years.

Mathematica is my go-to option for checking and simplifying symbolic maths. Need to solve a system of seven differential equations? Done. Forgot which integration trick to use or don’t feel like differentiating a variable that’s in a deep nest of functions? Done.  Need to factor a giant polynomial or simplify a hideous combination of arctangents and hyperbolic cosines? Done and Done. It’s also pretty good at quickly rendering 2D and 3D plots and animations. The interactive manipulation features have saved my bacon many a time when I’ve had trouble understanding how a function with more than 2 independent variables behaved. Those of you who think that Maple can do everything Mathematica can do are sorely mistaken. While it’s possible to customize things, it’s pretty frustrating and difficult to make anything look nice enough for print (I sometimes wonder if they borrowed some graphic design concepts from MS Excel 😉 ). Plus, the matrix toolbox, I/O, and memory management systems seem to be something of an afterthought, rather than a central design principle. So I almost always tend to use Mathematica as a kind of supercharged engineer’s notebook — great for sketching models and getting intuition on their behaviors, but not so good at presenting or processing lots of numerical data.

For those numbers-processing/plotting tasks, I typically turn to Matlab when it’s available. This is going to be a little bit controversial with some folks, but I actually love how Matlab tries to force you to re-think your programming task to phrase it in a “matrix way.” I say “tries to force you” because the interpreter has pretty poor performance compared to other languages if you phrase the programs in the standard iterative way. However, if you take the time to “think in Matlab,” you’ll reap more than extreme improvements in performance. You’ll be better slated to formally represent and more accurately understand your data processing procedure. The “heavy lifting” in many of my scripts is done in 15 lines or less, and can be easily described as a matrix equation of some form or another. That makes the writeup look a lot more professional, and makes the procedure easy to replicate. As far as presentation goes, the default graphing capabilities of Matlab are reasonably good for most cases, but can still require some fine-tuning adjustment in softwares like Inkscape before they’re print ready.

When Matlab isn’t available, I pull up Octave. In my experience, Matlab continues to be significantly faster than Octave, but generally the functionality and syntax is pretty consistent between the two. Octave doesn’t have any of the fancy GUI interfaces that Matlab does, but the only ones I find myself missing very much are the disttool and the sisotool —  but that’s just a convenience thing. Those were meant to be aids anyway, not crutches.

To those of you who are furious that I haven’t mentioned Python: I’ve used MatPlotLib+NumPy to generate some beautiful and customizeable plots in the past, but in the end I missed the “workspace” feel afforded by Matlab too much. I know Python has that sandbox mode, but it’s not quite as easy to seamlessly switch back and forth between what’s in your script file and what’s in the sandbox and what’s on the graph. Yes, I do realize just how gorgeous the MatPlotLib plots can be; but for the time being, I’m happy giving my Matlab plots a little post-processing makeup instead of the plastic surgery that they probably need.

Downloading and formatting historical wind generation and load data

This process refers to the Midwest ISO, but I’m sure other ISOs have a similar process. Thanks goes to Jason Watters at MISO Client Relations for pointing me to the data.

  1. Visit MISO “Library” page.
  2. Click “Market Reports
  3. Choose “Summary,” from the Report Type field.
  4. Select “Include Archived Files.”
  5. The Report Name “Daily Regional Forecast and Actual Load” gives you access to the load data.
  6. The Report Name “wind” gives you access to the wind generation data.

The wind data should be easy enough to download and collate by hand and then import into matlab or R or whatever.

Regarding the load data, it’s kind of a pain to download the ~1000 (and growing) files by hand, so I highly recommend the DownloadThemAll! firefox extension for this. Moreover, the format of this data is somewhat clumsy to work with: separate excel files for each day (containing hourly predicted and actual load information for the previous day and only predicted load information for the day itself). I used the following steps to get all the hourly data together, in a similar format as the wind generation data:

  1. Make sure you have OpenOffice installed.
  2. Download PyODConverter, and read the instructions on the website.
  3. Write a script to run the convert command on all files. Just to peeve my unix friends, I did this as a windows batch script: loopFiles.bat 🙂
  4. Write a script to concatenate the files and remove the unwanted information. In this case, I used Cygwin and piped together a couple of bash commands to do the job in a snap:
    head -q -n 31 *.xls.csv | grep ",,," -v | grep "Market Day" -v > winddata.csv
  5. Depending on your needs, you may want to combine the date and hour fields into a single timestamp. This can be done a number of ways, but it’s easy enough to use Excel or OpenOffice to do the formatting for you. Remember, the numerical representation of dates is in days.

And there you have it! Years of hourly wind generation and grid load data at your fingertips.