Jared Knowles

A case study in agentic data science: building the Census of Governments API Six weeks, roughly 100 human hours, 167 hours of agent time and 6.88 billion tokens went into turning 56 years of Census of Governments finance data into a queryable API. Here is what that actually cost, measured fro... 🔗

Today I had to take my server out of its basement wall mounted home and work on it in my home office. I am replacing a failing hard drive, repairing a failed cooling fan, and adding some additional drive capacity. I'm not happy that drive prices have jumped 75% per TB or more, but I do enjoy g... 🔗

Fall foliage reflects off the Sudbury River on a beautiful fall day. Though we could use some rain! 🔗

Work with Crystal if you can!

Checked out the Saugus Iron Works today, America's first iron works. It was great late afternoon autumn light to catch striking contrasts like this shadow of a waterwheel against the old wall of the refining house. 🔗

We went to the one night only light display at Garden in the Woods. The hills had eyes. There were glowing branches. And lots of other creative fall themed light displays. 🔗

We built a free API for 56 years of state and local government finance data with an AI agent, and I studied the transcripts. Here's what I learned: the agent put in 8.4 to 10.9 hours for each hour of mine during build weeks, and that held as work got harder. 🔗

Preliminary 2023–24 school arrest estimates, published as release 2026.1 Our Bayesian estimates of school-based arrest rates now cover 2023–24 and New York City, in a preliminary update not yet vetted like our white paper's 2025.1 release, which stays available, unchanged.

We drove up to Lowell for a free tour of the Lowell Cemetery which was a fascinating tour through the industrial and cultural history of the city amidst beautiful fall colors. Edgar Allan Poe spent some time in Lowell and those connections were a big theme of the tour. 🔗

Check out this amazing piece of public scholarship! I’m on the record as really disliking the NBER approach to paper releases, but I’ll make an exception here, because the authors have translated their research into the conclusions that matter to the public most. It is just a fantastic example... 🔗

Brennan Wedl put out my favorite debut album this year and has an incredible voice musically and lyrically. It was an absolute treat to hear her play her album in full tonight! 🔗

Seizing the passport, phones, and computer of an opposition lawmaker is straight-up authoritarianism. There is no other way to describe it. www.wsj.com/politics/pol...

I have to admit self-hosting is probably not cheaper than the cloud. One of my 4 12TB hard drives is failing and with skyrocketing disk prices, I could probably get 3 years of 12TB of cloud storage for the same price. Privacy is still worth it, but it’s going to hurt to pay up! #selfhost

Fantastic Cat played an incredibly entertaining set. The band members rotate instruments and each is a great vocalist. Fun songs, good vibes, poignant lyrics. 🔗

Sturgill Simpson decided to teach us the definition of face melting guitars at TD Garden tonight. 3 hour set. No opener, no encore, no breaks. Absolutely incredible. 🔗

Houston has a museum district and it means it. Affordable access to all kinds of art is one of the city's best features. 🔗

Now on jaredknowles.com/… we are fully fediverse enabled. You can see likes, shares, and replies right on the posts. All powered by webmention. How cool! 🔗

I learned a lot doing the retrospective on my Census of Government LLM development sprint. I tried to share most of it in this article, what worked, what went wrong, and how I am changing my work with LLMs moving forward.

Fall day at the source 🔗

It is always great when our community partners run with our work at Civilytics. Housing is a Human Right OC is an amazing organization. They show up tirelessly for funding housing first policies and budget transparency. I am proud to have helped them with this toolkit. 🔗

The Houston Museum of Fine Arts is a series of buildings connected by tunnels, because it's super hot there, and each tunnel has these really cool light installations in them. This purple one was just stunning. 🔗

Using my Hermes agent via Telegram on my phone to respond to decisions needed by my herdr Claude sessions on data pipelines I’m building to extend the new public data tools I’m building feels like Star Trek. Computer rerun the simulation accounting for measurement error.

Fall walks 🔗

When I build our Census of Governments API I wanted to use it as an exercise to learn and understand coding harnesses. I've been documenting what I learned with evidence. The key takeaway is you can't take your eye off the ball!

I’ve been doing education data work for a long time and somehow this never got on my radar: “Since 1979, the Western Interstate Commission for Higher Education (WICHE) has published Knocking at the College Door: Projections of High School Graduates. This analysis projects the number of high sc... 🔗

This interview with Sylvester Stallone is wise, emotional, and gripping. And, it really made me want to read his memoir. I knew only bits and pieces before (that he banked on himself for Rocky, didn’t like Arnold). But from his childhood to adult regrets, its an astonishing read. 🔗

Since America.gov relies on govt websites, it is mostly reliable, but where the Trump administration has corrupted those underlying sources of information (like public health), it is not credible. bsky.app/profile/dand...

I'm really excited about what we are working on at Civilytics these days. It feels like when I first learned how to make data visualizations again. A whole new paradigm for contextualizing data for people. Bring your questions and values and explore what matters to you.

Needed a walk in the sunshine today and was rewarded by the beautiful colors breaking through this street tree just up the road. 🔗

A minor point that is not important, but when did we switch to using German capitalization of all nouns? Ich habe diese Notiz vermisst. But wow, just wow.

In the last nine months, producing code stopped being my primary task. Coding agents got good enough that recalling an obscure R function is no longer a skill I need. Two things to build instead, from posit::conf: subject matter expertise, and software engineering practice.

Spent more time than I want to admit cleaning out dust and dead bugs from my server enclosure today. I’ve had some high temp alerts lately and I’m concerned about one drive in my ZFS pool so I’m hoping adding some airflow will keep things running until HDD prices come back down.

Why hello there! Today was a great day to get out and ride. Perfect weather and the foliage was looking sharp! 🔗

Took a ride today along a rail trail and had some stunning river views and this quintessential New England scene. That's me on the bike. 🔗

Amen

As someone who has immediate family struggling with mental health disorders, this profile was compassionate and very real feeling. It is hard to describe the ups and downs, day by day, but Brendan Quinn’s writing nails it. Much 💝 to Duncan and his family. 🔗

A really interesting "meta-analysis light" approach to summarizing evidence. It reminds me of work eliciting priors, which would be helpful to local evaluators as baseline before doing internal evals. Context matters so much in education as do the biases of the experts polled.

Took some time today to get out in nature and see some of the great open spaces near my own backyard. This is the Great Meadows Wildlife Reserve!

Saw Brandon Flowers, whose new album Thrasher is great, was playing down the road from me tonight. Grabbed a last minute ticket and hopped on my bike and it's time to rock!

Tarrant County's 300-page budget landed six days before the only hearing on it. A speaker gets three minutes, cut to two if 30 people register and one if 50 do. The better attended the hearing, the less each person can say. We drafted 10 comments so the group could make 10 arguments.

This article in the Economist gets to a layer of AI discourse that seems completely absent. Workers by the thousands are using AI to file administrative claims against their employers in Britain, putting a previously overly burdensome legal remedy in their hands. Sorry for the paywall, but thi... 🔗

Comment on Education Department General Administrative Regulations (EDGAR, 2026) The proposed changes in EDGAR would harm small businesses like mine and hurt innovation in education research. Register your opposition to the further politicization of education research today.

Maureen Tkacik wrote the heck out of this piece which explains the collusion, consolidation, and consumer exploitation at the root of the censorship of Macklemore. The roots run deep and old, but there is hope that the public is showing we’re fed up! 🔗

Great system, no notes www.propublica.org/article/scho...

Scenes from Posit Conf

One of my favorite talks of the whole conference!

I loved this talk. Mirai reinvented my workflow and mori looks like another huge level up! Thank you

Incredibly impressive project. Excited to collaborate with non-Quarto users much more seamlessly!