Six months reading every Italian municipality

On April 1st I opened a new Rust workspace.

No, it was not a joke, even if the date was suspicious 😄

The plan was very small: learn Rust by scraping the public notice board of one municipality, Scicli, a town 15 minutes from my house. One town, one website, one crate, that’s it.

Six months later the numbers are these:

• 7,896 Italian municipalities, all 20 regions, plus unions of towns, health agencies, parks, provinces… 9,617 public bodies in total
• more than 5.1 million administrative acts
• 17 Rust crates in the workspace
• 2,161 commits
• three products and one new job title

So… something went a bit out of control 😅 and there are too many things for one post, so this is the first of a small series, mostly for future-me, because I already forget the details of things I did in May.

What an “albo pretorio” is

In Italy every public body has an albo pretorio, an online notice board where they must publish their acts: council decisions, spending approvals, tenders, public works, the money for the town festival… everything.

It’s public by law and anyone can read it, at least in theory.

In practice every municipality buys its website from a vendor and every vendor built its own notice board, in its own way and with its own bugs, so there is no standard and no API. If you want to know what happened in 7,896 towns you have to open 7,896 websites, one by one, every day. Good luck! 🙃

So I built albopretorio.io, a crawler that does it for you and one place where you can search everything.

About Rust (and about honesty)

The original idea was a strict “learning mode”. I wrote a file with the rules for Claude: be my Rust tutor, explain the compiler errors and do not write code for me. It even says which HTTP client to use (ureq and NOT reqwest, to avoid async at the start) and that .unwrap() is fine “for now”.

The funny part? That file is still in the repo and it still says that we are in “Phase 1” and that “Lorenzo is writing the crawler” for Scicli 😁

The truth is that the Scicli scraper was already committed on day one and on day three there was an LLM reading the PDFs of the acts… so, let’s say the rule didn’t last long.

When the project became bigger than a toy I switched to the same way of working I described in the Astro migration post: I decide what to build, the agent writes a lot of it, I review, I test and I fix what it doesn’t see.

Did I learn Rust? Yes, but not in the way I planned. I learned to read it very well, I learned ownership by fighting with it on real problems and I learned why Result and ? are better than try/catch, but I didn’t write every line and I’m fine with that, because the goal changed on the way.

What I would say to anyone doing the same is that the agent is great at adding things, BUT you must be the one who checks them, especially the numbers.

What I learned (so far)

Starting with one town was the right call, Scicli was small enough to understand every page of it. The second town taught me more than the first, because it looked the same and it wasn’t.

About numbers… it happened again while writing this post! The first draft said 1.7 million acts, taken from notes of July without checking, today they are 5.1 million. Same story with visitors, empty municipalities and costs: every number I shared too fast was wrong.

And the infra is almost boring: some Rust binaries, one Postgres and a couple of machines on OVH, nothing fancy. It holds 5 million acts anyway 🤓

Closing

Six months ago I wanted to learn a programming language.

Now I know which vendor runs the notice board of a town of 300 people in the Alps, how many sagre there are in Puglia in August and which public domains are selling pills 💊

Not what I planned, but I’m happy it went this way.

In the next posts: the zoo of vendor platforms behind the notice boards, the festivals hidden in the bureaucracy, a product that nobody bought (yet) and my new role at Welance. One at a time. 🙂

---

Cover: art from Book Burning (Magic: The Gathering, Judgment, 2002) by Dave Dorman. Magic: The Gathering is © Wizards of the Coast.