I had always thought that the Thomas database of U.S. Congressional behavior was cranky. If I search for a bill, for example, the search results don’t provide permanent links to the bills in question. These permanent links exist, so that would be possible for Thomas to do for me. But no, they provide temporary links that expire within a few minutes, so that if I spend too much time perusing the list, I can no longer click through to information for bills. A link to a browsable list of bills before the House or Senate should be available on the Thomas front page, but it isn’t. To get to that list, I have to click on a link for “Advanced Search,” then a link for “Search Bill Summary and Status,” neither link of which describes what I’m actually looking for. Why, it’s enough to make me…
… oh, all right, just get mildly irritated. I’ve held the Thomas system for the U.S. Congress in low regard simply because I haven’t looked thoroughly at what else is out there. Take the Maine State Legislature database, for instance. I’ve been working to understand its logic, and working, and working. The system tunnels its way in, around, and back through a series of websites: janus.state.maine.gov/legis, mainelegislature.org, and legislature.maine.gov/bills, and it’s hard to tell where one ends and the next begins.
Currently, I’m looking for information on cosponsorship of bills. In the state of Maine, a piece of legislation may typically hold up to ten sponsors (and atypically a few more than that). Head to a list of bills and pick a bill. I’ve picked LD 1496 (LD stands for “Legislative Document”). The main page for LD 1496 contains links to the bill’s text, committee status, amendments, divided report, and sections of Maine state law affected. These pages of information are referred to by the index number 1496, related to the bill. But to actually print the full text and set of amendments on screen, you’d need to reference the number 573, for SP 573, the “Senate Paper” number for the legislature. As long as you keep the connection between SP 573 and LD 1496 in mind, you’ll be fine. But what if you want to know who all the sponsors for LD 1496 are? Click on the “Chamber Status” link, a page on which various bill information, including a list of sponsors, is referred to by a unique “ID” number of 280033996.
280033996? There’s no reference I can find after hours of searching to indicate what this ID number “280033996″ is, what it references, or even how it increments from bill to bill in the Maine state legislative database. With checking from bill to bill, I can tell you that those ID numbers don’t simply increase as LD numbers increase; the unique ID number for LD 1495, for instance, is “280033956.” Sometimes bills with higher LD numbers have lower unique ID numbers. There’s no web page I can find that shows what the particular correspondence between LD number and ID number is for the set of all bills.
This might sound like a trivial problem, but from my point of view, it’s not. While all the information on bills is nominally in the public domain for residents of the state of Maine to review, the lack of a consistent connection between numbers referring to bills means that any attempt to analyze patterns of behavior in the legislature, across all bills, is stymied. Typically, I’d write a little program asking my computer to visit a series of these government websites, indicated by some systematic web page address containing a predictable number, and pull out the necessary information I’d need to tell you…
Who is cosponsoring more or less bills,
Who is cooperating more or less with which other state senators or representatives,
Which bills are gaining support from what kind of legislators,
…and so on. Those are questions about patterns of behavior in a legislature, and unless I can predictably gather information for various bills, I can’t provide the forest-for-the-trees analysis requiring systematic data collection for the thousands of bills that exist.
A call to the computer department of the Office of Legislative Information of the state legislature confirms that the unique ID number is generated in an a way that doesn’t predictably increment, by a software program called LawMaker. The people in the IT department were very helpful in confirming this to me, and in confirming that the information system being used has disjunctures: between the information technology people who maintain the mainelegislature.org website, for instance, and the people in the House and Senate who enter legislative data on the other hand.
There are indications that the frustrations of the Maine state legislature database aren’t due to bad people confounding public access out of malice, but rather to an old system that good people must push to make as workable as they can. An indication of the data entry nightmare faced by the tech folks in Augusta:
The texts of bills available on this web site are translated from their native WANG Word Processing format to Microsoft Word, and then from Word to HTML via a collection of Word97 macros.
Unfortunately, due to the complexity of this process we cannot guarantee that the text available on the web in any format is 100% faithful to the original…
You read that right: these folks are still stuck with Word97, and prior to that WANG Word Processor (or WANG emulation) entry. The last time I heard reference to WANG word processors was in the 1993 political comedy Dave.
I know that I’m not talking about a problem as great as world hunger, but I insist it’s not a trivial one: the difficulty in collecting information is proportional to the amount of understanding people can gain from that information. When questions are made harder to answer, they’re less likely to be answered. Greater understanding of the behavior of the Maine state legislature has become possible over the past decade as bill information has moved online; even greater understanding will come when there are enough resources to iron out idiosyncratic databases into a more systematic format.