Operations Research and gerrymandering

INFORMS blog challenge for January involves Operations Research and politics.  While public debate on politics will never change perhaps one thing that can change is the involvement better decision making to solving some of the political discourse.

One area of politics is the topic of gerrymandering.  For those that are not familiar to gerrymandering it is the process of resetting electoral boundaries for voting purposes.  Gerrymandering is a hot topic around any election because it is usually the party in power controls the rights to reset the electoral boundaries.  This leads to an obvious advantage to the party as they can maintain a seat in a legislature with setting boundaries based on past voting behavior.

Operations Research can be a valuable asset to the process of redistricting.  In fact Operations Research has been very much involved in redistricting for at least 50 years.  Decisions to draw electoral lines can follow any number of constructions including demographics, population centers, municipality boundaries or industry types.  As information abounds more freely there is more opportunity to use it for decision making.  It seems every new census brings more available data.  The growth of the internet has allowed information to be available more openly.  Opportunities should grow in Operations Research to provide redistricting decision makers the information for informed analysis.

Perhaps one of the better uses of Operations Research could be the ethical context of the gerrymandering debate.  I have often heard it debated that Operations Research may have created the politically polarized country we have today.  The same tools of Operations Research could be used to allow transparency in the redistricting process.  It could be useful for citizens to know how probable or likely outcomes of elections based on redistricting suggestions.  Websites like OpenSecrets.org shows how money influences party affiliation and elections.   Perhaps similar websites can emerge on electoral districts and the legislation that helped create them.

The debate of gerrymandering will last for centuries I am sure.  I believe Operations Research can play a vital part in the debate.   Information is more open and easy to access than ever.  Let's use that to our best ability and help inform the voting electorate.

Statistics blog roundup from the Internet

For those of us that want to keep up with the latest in the world of statistics here is a great list of 40 statistical blogs on the internet today from bschool.com.  Some of the blogs even include R (link) which is my favorite statistical computing environment. 

I have a few that I read on an occasional basis.   Those include

R-Bloggers - an R blog aggregator that is very useful for R enthusiasists

Revolutions - blog by David Smith of Revolution Analytics, an enterprise ready R support and developer company

The Numbers Guy - Wall Street Journal blog about, you guessed it, numbers and statistics.

As you can tell I have been doing a lot of statistical work lately.  Most of my efforts have been in the realm of predictive analytics.  These blogs have come in useful in finding valuable references for predictive analytics.  One book that is often mentioned in these statistical blogs is Applied logistic regression (Wiley Series in probability and statistics).  I highly recommend this book as a statistical reference for logistic regression analysis.

Applied Logistic Regression takes the reader through an understanding logistic regression model and its usage for modern prediction analysis.  The book starts with and understanding of log odds and introduction to the logistic regression model.  Then the book goes into detail of the model parameters, coefficients, and estimation.  Strategies for model building and case studies are also referred in this book.

Video of Joy of Stats by Hans Rosling

The Joy of Stats narrated by Hans Rosling was just produced by BBC and shown to their audience.  Hans Rosling via gapminder.org was kind enough to post the full hour video of the documentary about joys of statistics.  The video is posted on YouTube and is available to anyone.

http://www.gapminder.org/videos/the-joy-of-stats/

Hans Rosling's passion for statistics is infectious.  He definitely has a joy about him that persuades the viewer to really enjoy finding new and invigorating ways to explore data.  Now for me this is not hard to do as I love data and analyzing.  Yet for many in the world mathematics, let alone statistics, is considered a universe all unto its own that they dare not search.  Hans breaks down that barrier with The Joy of Stats.  No matter your educational interests or background I find it very hard to ignore his plea that statistics is not boring, and dare I say it, sexy. 

If you are interested in this video as a eulogy to statistics you would also enjoy Dr. Robert Lewis's essay on Mathematics.  Both of these works explain how a world without number analysis is merely a world not worth living.  There is so much to explore in so little time.  I am so happy that I decided to take a career in Engineering and Operations Research to help the world one datum at a time.

Top IEORTools Blog Articles of 2010

According to my Google Analytics account these are the top posts by pageview for the 2010 year.  Since this is an analytics based website I thought it would be appropriate to provide some data.  So here is a sort of year in review for IEOR Tools.

1.  Favorite Operations Research books from OR-Exchange

2.  R references for handling Big Data

3.  IEORTools Tutorial: Learning XML with R

4.  My 5 Favorite Operations Research Blogs

5.  Where to find good data sets

A lot of the pages had to do with using the statistical computing software R.  I'm also a contributor to the R-bloggers website so that has a lot to do with the traffic.  I'm excited to see what 2011 will have in store for the OR blogging world.  Happy New Year to the Operations Research community.

An essay on Mathematics and education

I just read an essay on Mathematics by Dr. Robert Lewis named "What Math?"  This has to be the single best essay I have ever read on Mathematics and why it is so important.  I can imagine Dr. Lewis has been peppered with cynic criticism about understanding math just as much as any of us in our analytical profession.  Dr. Lewis does a superb job of explaining the essence of why those of us that use math love it so much.

Educating Math to our newer generations is definitely a concern.  I really like how Dr. Lewis explains that education is not just about transfer of information but the understanding of underlying principles of specific knowledge.  The parables are a very clever device to relay those principles of math. 

I also love how he portrays Math as not just a device for the technologically minded but also for the liberal arts.  Dr. Lewis conveys that Math is not merely knowing numbers but the processes of finding solutions.  My own example is when people often ask me how I am so good at math.  I usually tell them its just like learning a language.  Once you understand the language and are fluent then you can start applying it in everyday life.  Math is a language to learn just as much a foreign language.  It may take some time to learn but it will take a lifetime to master.

I highly recommend reading this essay.  I also recommend saving this essay for our future generations, teachers, educators, family members, and friends.  This essay can be used to help bridge understanding that may be missing from our own words. 

New OpenOpt/FuncDesigner quarterly release

New OpenOpt and FuncDesigner quarterly release is out: 0.32.

OpenOpt:
* New class: LCP (and related solver)
* New QP solver: qlcp
* New NLP solver: sqlcp
* New large-scale NSP solver gsubg. Currently it still requires lots of improvements (especially for constraints - their handling is very premature yet and often fails), but since the solver sometimes already works better than ipopt, algencan and other competitors it was tried with, I decided to include the one into the release.
* Now SOCP can handle Ax <= b constraints (and bugfix for handling lb <= x <= ub has been committed) * Some other fixes and improvements

FuncDesigner:
* Add new functions removeAttachedConstraints, min and max
* Systems of nonlinear equations: possibility to assign personal tolerance for an equation
* Some fixes and improvements (especially for automatic differentiation)

See also: Full Changelog, Some Applications, Future Plans

Where to find good data sets

O'Reilly Media has been a big advocate of Open Data and believes that is where a lot of computing is going to be headed in the future.  I think they are definitely on to something.  Yet the future could be now.  There is a lot of opportunities to find good data sources immediately.  One of my favorite blogs, OReilly Radar, has an article by Edd Dumbill on Where To Find Data.  There is plenty of good data available on the internet for download to explore and mine new information.  These places not only offer great sources of data but many of them offer an API to allow quick and seamless access.  Below is a link summary from the article.

Freebase

An all-things graph database.  The website focuses on trends of certain cultural and interest topics.

Amazon Public Data Sets

Amazon is probably considered the cloud computing mecca next to Google.  Amazon Web Services offers a lot.  One of which is storage of public data sets.  They offer a huge variety of public data.

Windows Azure Data Marketplace

Surprisingly Microsoft has an open data protocol data source.  This data market offers quite a few points of interest data sets.

Yahoo Query Language

YQL is an interesting API that is very similar to SQL.  YQL is essentially a language that allows to grab data from cloud services.  This could be very handy to grabbing data quickly and dynamically.  YQL offers to connect to a lot of data sources as well.

Infochimps

Infochimps is a data marketplace warehouse.  They offer to host, sell, and distribute data sets.  Some of their data comes at a cost but a lot of their data is free as well.  This is an interesting startup and will be very interesting to follow their growth.  Also there is a new Infochimps R package that uses their API to gather data and process Infochimps data.

DBpedia

DBpedia is a wikipedia for data sets.  In fact the data itself comes from Wikipedia. 


Some other sources not from the article include the World Bank open data and the U.S. Census data.