Top 50 Statistics blogs of 2011

TheBestColleges.org published their list of the top 50 statistics blogs.  This is a really good resource list of statistical analysis and news.

Data mining the Federal Reserve

The Federal Reserve now has the ability to have its data programmatically retrieved.  The St. Louis Fed Web Services allows programmers and data scientists to retrieve key economic data from their libraries.  I have not had a chance to peruse the site at all but this can be a really interesting source of data.  The age of Open Data is really upon us.  This can lead to some really interesting research for professionals and amateur scientists.

Machine Learning for everyone

Well maybe mostly everyone. Have  you been interested in gaining knowledge in the latest craze of artificial intelligence and computing?  Then go no further than Stanford's Machine Learning course which is now open enrollment to everyone!  Andrew Ng is back to provide the world with knowledge about Machine Learning for the entire masses. 

Per Stanford's website, Machine Learning is data mining and statistical pattern recognition.  Mostly it is applying mathematical and statistical methods to draw out information behaviors from data sources.  So do you want to invent the next Netflix, Amazon or Google?  This is the course for you.

If you do not want to enroll in the Machine Learning class you could always watch some of the older lectures online.  Andrew Ng provides plenty of information from past lectures with student contributed projects.  The CS 229 website is worth a look for a punch of Machine Learning related resources.

Data Driven Success in Professional Baseball

An interesting article from Data Center Knowledge about the presentation Paul DePodesta gave at the Strata Summit.  Paul DePodesta is known for bringing mathematic and analytical know-how to Billy Beane and the major league professional baseball team Oakland Athletics.  His story was accounted by Michael Lewis in the book "Moneyball" and is being portrayed with the same name on the big screen opening this weekend.

I really liked this quote from Paul in the article.

We didn’t solve baseball. But we reduced the inefficiency of our decision making.

Is that not the sort of things that an analytical professional or an Operations Researcher ultimately tries to do?   Operations Research is not the art of creating anything new.  It is the art of creating existing things better.  All decision making is inefficient to some point.  Even the right decision can be inefficient on some level.  Decisions are full of balancing acts of constraints and feasibility.

Also this proves that no industry or organization is absent of a need for efficient decision making.  Even baseball can us a dose of improved decision analysis.  Whether is scheduling the league or determining the best pitcher for their value.  Sports has definitely come into their own with decision analytics.  I'm eager to watch Paul's career and wonder if analytics is taking it to the next level.

OpenOpt Suite 0.36

New release of the free BSD-licensed software OpenOpt Suite is out:

OpenOpt:

* Now solver interalg can handle all types of constraints and integration problems

* Some minor improvements and code cleanup

FuncDesigner:

* Interval analysis now can involve min, max and 1-d monotone splines R -> R of 1st and 3rd order

* Some bugfixes and improvements

SpaceFuncs:

* Some minor changes

DerApproximator:

* Some improvements for obtaining derivatives in points from R^n where left or right derivative for a variable is absent, especially for stencil > 1

See http://openopt.org for more details.

Physicist cuts airplane boarding time in half

I have always been fascinated with the airplane boarding problem.  I wish I was in the airline industry because I would love to tackle this problem.  I used to travel a lot for my job and I would marvel at how inefficient the time  it took to board an airliner.  My first inclination is to redesign the plane (and the airport jetway) to include exits at middle and rear of the plane to go along with the forward exit.  Yet I never put my ideas to paper and tried to calculate efficiency gains.  There have been a lot of ideas try to find the optimal boarding arrangement.  Would you believe that random boarding, i.e. Southwest Airlines, is a more optimal boarding procedure then the current row assignment method?

Yet a physicist from Fermilab, Jason Steffen, did have some interesting ideas to improve the existing airplane boarding procedures.  By using Monte Carlo simulations to measure efficiency and test his ideas he was able to improve airplane boarding by as much as half the time.  From the article, his methods were to using sections of window seats first but alternate aisles so passengers would not interfere with each other.

This is a very clever idea.  Yet I found one flaw that may not have been assumed in his study.  I've noticed that overhead space is a premium for passengers, especially for business travelers.  Business travelers often bring two carry-on bags.  These bags tend to fill up the overhead bins rather quickly.  When the overhead bins fill up then passengers have to search in the aisles looking for available space for their bags.  This creates a bottleneck and queues develop for the other boarding passengers.  It seems to me that Jason's study makes an assumption that all overhead bins would be available at time of boarding.  If in fact alternating rows are used in his model than overhead bins might become filled to capacity before passengers board and create more bottlenecks.  Its just one theory that would be worth investigating before Jason's procedures are implemented.

I applaud Dr. Steffen's studies and finds in the airplane boarding problem.  It is a fascinating problem as most of us have encountered airplane boarding from time to time.  For more information on his methods you can read about Jason's work airplane boarding, which is very fascinating, on his website.

What did we learn from the Space Shuttle program

The NASA Space Shuttle program ended this past week with STS-135.  I still remember watching the shuttle launches as a boy.  They filled my head with dreams of space exploration and new discoveries.  As I grew older I took an interest in engineering and studied to be one in a university.  There I discovered the enormity of the engineering marvel that was the Space Shuttle program.

Discover published an article this week on what was the debacle of the Space Shuttle program.  A lot of good and interesting points made by Amos Zeeberg in this article.  The Space Shuttle was originally designed to be a cost effective way of getting man and technology into space.  The program definitely did not deliver on that promise or projection.  Also the Space Shuttle was considered to have only a risk of failure of 1 in 100,000.  I don't know if that is remotely true.  As we unfortunately know the true risk of failure was 2 in 135.  Space travel is risky no matter how it is done.

So from an engineer's point of view, albeit one that was not involved with the space program, what can we really learn from the Shuttle Program.  I believe applying Industrial Engineer and Operations Research principles we could come to some conclusions.  I don't personally think the Shuttle missions were a total debacle.  As an Engineer there is always something to learn even if there is a failure.  Edison said it best that he didn't fail 1000 times trying to develop a light bulb, only he learned 1000 ways on how not to build one.

Firstly, risk needs to be measured from a micro and macro perspective.  There are many systems that lead to failure.  Each system has a life all of its own.  The risk could be as simple as an O-Ring to as complicated as a practical study of landing on the Moon.  Risks can be measured and weighed from different perspectives of time, cost, and quality of delivery of promise.  When all risks are measured than perspective can be put into place as to delivery of a promise.  Perhaps the Shuttle program didn't deliver on all promises.  Yet it did prove many things that reusable vehicles were ahead of its time.  We can learn a lot from the Shuttle Program on examining risks of promise and making sure that we evaluate different objectives and goals.

Secondly, engineering and management should be a cultivated relationship that needs to understand each others' strengths and goals.  Engineering has the design in its best interest.  Management has the mission in its best interest.  The design and mission are unique and have there own set of goals.  Yes there are going to be risks weighed in both the design and mission.  The complexity is when merging the risks of the design and mission together.  The magnitude of the NASA Space Shuttle Program magnified the relationship between engineering and management.  The best and the worst was brought to light.  The engineering marvel of creating a reusable vehicle is magnificent.  The managerial feat of sending man into space with a reusable vehicle on more than 100 missions is not insignificant.  The importance of merging design and mission together was a great learning experience with the Space Shuttle program.  We have already seen fruits of that success.  Missions to Mars and beyond the Solar System have proven that success.

The NASA Shuttle Program was not an outright debacle.  There was a lot to learn from the process.  No it did not deliver on all initial expectations.  Yet it did deliver on this young boy's dreams of discovery and knowledge.  Once an Engineer, always an Engineer.  I hope that we will never cease to learn and improve from our failures.