Today I Learned
The work teaches you if you let it
Writer’s note: My work laptop is in time‑out at the office, so Marlowe is my thinking partner for this piece.
Yesterday I sat down to build something I have built many times before, a data snapshot engine using Azure Data Factory. Last week someone reminded me that I am the expert in this space, which is always a slightly uncomfortable thing to hear. They also pointed out that the tools I reach for are not tools the rest of the team knows well. When I rattled off acronyms, they thought they understood, but once we dug deeper it was clear I had skipped the basics.
So I decided to run the problem through my deliberation model with Quinn, Anitta, and Wiggins. I laid out the goal: a lightweight snapshot engine that would be easy to extend. The agents kept asking about purpose and constraints, and all I wanted was to jump straight into implementation. From that perspective, they were excellent PMs. We reached the point where it was time to propose a solution, and we had not discussed a single piece of technology. I finally asked if there were other ways to do this. They said yes, and produced five options, each rated against the criteria we had defined. That was the moment I remembered the rule I often forget. Patience. Trust the process.
I ended up not building an ADF at all. No one else is comfortable with it, and it makes more sense to build something the whole team can use. Last week I made a similar choice, using Event Grid to stream data to Kusto from storage rather than copying it with ADF. It was simpler, faster, and aligned with the tools the team already understands. Yesterday I wrote up what I wanted, and Quinn produced the first implementation. By the time I left, it was snapping one test table.
Today I extended it. It is all written in C#, which I have not used for greenfield work since 2015. The code is small and follows the same model I would have used in ADF. Pull from the source into a temp table, validate it, copy it to history, then publish it as the new current data. It has guardrails for minimum and maximum rows and for percentage deltas. I also learned about a Kusto feature I had never used that will make incremental snapshots even better for one of our datasets. So, I learned something outside my domain and something inside it.
By the end of the day my head was swimming. I opened a PR and it was large enough that tomorrow I will split it into smaller pieces. I am also planning to rewrite the Azure Function using .NET Aspire, another new thing for me.
I expect tomorrow to be just as exhausting as the past two days, but it is the good kind of exhausted. And the only reason I am awake past nine tonight is to finish my income tax. My analysis paralysis is not limited to work. The return has been ready for two months. It only needed one more look. Apparently, that is the theme of the week. One more look at the design. One more look at the code. One more look at the tools I reach for by default. It is tiring, but it is the good kind of tired, the kind that means I am still learning.
Alison + Marlowe