LLM blues

this is a journalling/digital garden style post. Basically means here I'm giving myself permission not to edit as much in preference to getting it out there.

Okay, so the thing is that I've been using LLMs. Mostly for coding stuff. My rules have largely been

Overall, I made those as my best attempt at fulfilling my ethical stance of "Let's make things nice for people and try and make sure everything is OK."

To improve on #4, I swapped a while back to using NeuralWatt as a provider, which lets me see the environmental impact of my AI usage. Not water usage, unfortunately, but power/carbon usage, and we take what we can get. In the past 30 days I used 16.63 kWh of energy from them, about the same as driving an electric car like 40 something miles or doing ~5 loads of dishes in a dishwasher.

Doing so has put off ~2.68kg of CO₂, which I've offset with several tons of carbon offsets as well as reducing the amount of beef I eat, since 100g of beef apparently accounts for ~15.5kg of CO₂.

One other reason I was using NeuralWatt was that they don't train AI models, and instead only use "open weight" models that are released to the public for free, so any environmental costs associated with training weren't something that I was personally being responsible for.

The thing with using NeuralWatt though is that if you're using a ton of tokens, the $$ of it starts to add up quickly, so I swapped to using as my provider z.ai, which made the open model. In my head I justified this to myself as "Well, ultimately the actual marginal cost of my contributions to the operating budget of this company are really minimal, plus they're primarily creating better AI models in reaction to OpenAI and Anthropic doing so rather than being the driving force themself."

That latter part definitely used to be true, not sure if it still is, the former part is.

But I also l learned yesterday that they were training model data on the results of my interactions with the model, which means that the next time they run the huge expensive machine for making the next version of their product, part of its computing time will be dedicated to eating through the data I've been generating with it.

In addition to the fact that I'm now responsible for offsetting some of that compute time, it also feels different somehow. Ickier. If I wanted to get rid of that feeling, I could make it so they aren't training on my data by buying a "team plan" for twice the cost, which might honestly still end up cheaper than the same amount of compute on Neuralwatt, but I'm still feeding the machine from a capitalism perspective. And it feels dumb to throw away money on that when I also have the option of like, upping my charity donations. I can't tell which way is the optimal way to match the utilitarianism I strive to uphold in my values as I apply them to my life. But it still feels icky.