Choppy AI Thoughts

Posted , , ,

I don't know how to categorize this, it's a choppy collection of random and likely non-unique thoughts pieced together.

I don't care much for AI, and if I had to summarize my general opinion, it's likely to lead to an increase (potentially exponential) in the amount of garbage in the world. This is already observable in terms of the content on the internet -- there's little conceptual difference between old school blackhat SEO blogspam of yore and current-generation AI generated blogspam. The latter is slightly more useful, readable, and less spammy -- but it's close enough.

There are also some larger losses, both for individuals and at larger scale. These losses are more nebulous and abstract and don't seem to be a problem (yet, because it'll take time for the garbage to accumulate). This is, again, a very choppy attempt at summarizing some of those less-observable losses.

Individual Knowledge/Information Losses

Context Stripping

Simple problem, but observable right now: AI might give you passably similar answers as a search engine, but it also strips all of the context from surrounding answers (e.g., Stack Overflow at least has dates and comments). It's cool to get a neat summary, but the loss of commentary, dates, and all the surrounding data is still a loss.

That awful detailed context stuff is important for decision-making and building your own summary and conceptual understanding. You might not always remember and recall precise specifics of the context, but the details do slowly compile together and eventually congeal into a better "big picture" in your head...

Fundamental Knowledge Loss

... the loss of which seems pretty catastrophic, in terms of establishing fundamental knowledge. If you rely on AI to build your summary for you, it atrophies the muscles in your head relevant to putting all of the pieces together and gaining deeper understanding.

It's Sparknotes -- even if you can pass a test or Get Shit Done with the short and sweet version, should you, if it robs you of core knowledge? If you offload that work, you lose the opportunity to find these things yourself, think about why it's happening, and actually learn for next time. "Computer does it for me, why should I learn?"

Same principle as in school with "you won't always have a calculator" -- wrong if taken literally, while the underlying point about not understanding things if you don't do the work is still true. I hated doing the work for math and avoided it wherever possible, and that's (part of) why I suck at math today. (There could be a whole separate essay here debating the age-old "technology atrophies knowledge" thing we've been saying probably since the first proto-human picked up a rock to use as a tool, but for "brevity" we'll skip that.)

It's the same underlying principle that used to be touted in web development. "Don't learn the framework, learn the language. Don't learn jQuery, learn how JavaScript actually works." If you are working at a level of abstraction that hides the internal workings from you, you can move much faster -- at the cost of deeper understanding. Thus your ability to improve things or forge new pathways suffers.

Each layer of digital abstraction, no matter how well it is crafted, contributes some degree of error and obfuscation. No abstraction corresponds to reality perfectly. A lot of such layers become a system unto themselves, one that functions apart from the reality that is obscured far below. (You Are Not a Gadget, Jaron Lanier)

It's much harder to learn the slow way, it takes much longer... but knowledge is cumulative and involves a lot of slog-work through hard stuff. That's particularly troublesome if you're a newer developer or working on something that's new to you.

This is all also worsened by automation bias, a well-known phenomenon in which humans tend to trust machine-generated answers over their own perception.

Maintenance vs Creation

Short and sweet: at least when people talk about AI and code, they mostly talk about new. Add code faster, write code faster, churn out new things faster. Broadly speaking (implied: this is a generalization and won't hold true for everyone) people rarely talk about using AI to maintain, troubleshoot, and untangle existing code and systems. Most of the discussion surrounds "new new new shiny."

This is a problem for two reasons:

  • You can't always just rewrite the thing and start from scratch. Rewrites can be risky.
  • Maintaining, seeing how solutions actually survive in the real world, how to fix the things you got wrong on first pass is invaluable learning experience. "It's easier to write code than read it" essentially. Even when you can rewrite the thing, it's stealing away knowledge gained via maintenance, bit by bit.

I don't know that AI can't help untangle and maintain things, fixation on new is just an observation about discussion trends.

Broader Knowledge/Information (and Miscellaneous?) Losses

Loss of Public Knowledge Base

Similar to the above point regarding individual fundamental knowledge loss... there's also a loss of "public knowledge." You can't browse StackOverflow tags anymore, or just trawl around through random threads or "Wikipedia link follow" things. That sort of "random discovery" often leads you to something useful that you didn't even know you were looking for.

This is already the case where -- instead of having an open, public, and searchable knowledgebase -- information is locked behind private Discord and Slack servers, or otherwise "members only" areas of the Internet. Knowledge is often not as easily available to the public as it was. It represents a major loss in terms of publicly, openly browseable and searchable information.

You can still ask AI all the questions in the world -- but that's more like "suggested related content" at the bottom of a page, where your potential answers are more limited and were pre-selected for you.

And you have to know enough to even ask the question in the first place.

(This point is particularly choppy and abstract. I don't know how to phrase it much better.)

Mimetic Collapse & Consensus-Based "Expertise"

I also have difficult to articulate thoughts about "mimetic collapse". AI seems to be accelerating this further, everywhere -- but particularly in creative, artistic, and technical pursuits.

...the artistic obsession with novelty and experimentation, the primary obsession of modernism and so something like the default goal of artists for more than a century, has recently run aground. This turn from the primacy of the new does not stem from a choice to reject it, but because culture is truly spent, and can produce nothing original. The writer, Jason Farago, suggests that this doesn’t have to be a crisis, that there can be a lot of value in people continuing to iterate on the same old moves. I don’t really agree, but I also don’t have it in me to defend a core artistic principle in a culture that’s so inimical to my values.

Freddie is a hard read and his ideas tend to be buried in a lot of words. This is a particularly bad example of him using one example to speak to a much bigger idea, so my summary take is essentially he's saying culture itself is becoming endlessly recycling and reusing versus innovation and creativity. The loss of artistic and innovative creativity is an actual cultural crisis.

Widespread use of AI would only accelerate all of that. AI is essentially a consensus-creation machine, it doesn't provide new thought. It can help humans with new thought, but it doesn't inherently provide new thought of its own. It's generative and not truly analytical or intelligent. It's the same problem as described in various ways and formats for a really, really long time: reliance on consensus and the most obvious answer, and collectively running out of new things to say, do, and create.

Because that is, in fact, what AI as we know it is doing now: generating consensus on a pretty massive scale. It's only capable of generating the most likely string of words that constitutes an answer, things that have already been said.

Here is one activity where committee "expertise" is an obstacle. In a committee which must "produce" something, the members must feel a strong impulse toward consensus. But if that something is to be a map of the unknown country, there can hardly be consensus on anything except the most obvious. Something really bold and imaginative is by its nature divisive, and the bigger the committee, the more people are likely to be offended. (The Organization Man, William H. Whyte)

The distinction between first-order expression and derivative expression is lost on true believers in the hive. First-order expression is when someone presents a whole, a work that integrates its own worldview and aesthetic. It is something genuinely new in the world. Second-order expression is made of fragmentary reactions to first-order expression. (You Are Not a Gadget, Jaron Lanier)

Pattern exhaustion: A state of creative decline. Derived from archaeology, the term in its strictest sense describes excavations that over time reveal stylistic repetition rather than variation in artifacts such as pottery. (Wired)

Note that "AI struggles with controversiality" is not what I'm saying here. AI does struggle with controversial, undecided stuff that doesn't have a well-settled answer -- but that's not what I mean.

I'm actually talking about the non-controversial, well-decided stuff. Even "well-settled" areas of knowledge are probably not all that well-settled. I somehow doubt we've reached the end-game of knowledge to the point we can let machines do the thinking for us. Leechcraft and blood-letting were once well-settled medical science, and the guy who invented hand-washing prior to surgeries was lambasted by his contemporaries -- imagine if we hadn't continued new discovery and innovation past those points in history.

Reliance on the machine and its consensus-based answers seems a good way to worsen the creative innovation collapse.

The progress of AI in the closed and orderly world of chess, with instant feedback and bottomless data, has been exponential. In the rule-bound but messier world of driving, AI has made tremendous progress, but challenges remain. In a truly open-world problem devoid of rigid rules and reams of perfect historical data, AI has been disastrous. IBM’s Watson destroyed at Jeopardy! and was subsequently pitched as a revolution in cancer care, where it flopped so spectacularly that several AI experts told me they worried its reputation would taint AI research in health-related fields. As one oncologist put it, “The difference between winning at Jeopardy! and curing all cancer is that we know the answer to Jeopardy! questions.” With cancer, we’re still working on posing the right questions in the first place. ... “AI systems are like savants.” They need stable structures and narrow worlds. When we know the rules and answers, and they don’t change over time—chess, golf, playing classical music—an argument can be made for savant-like hyperspecialized practice from day one. But those are poor models of most things humans want to learn. (Range, David Epstein)

Other Losses and Negatives

But wait, there's more! ⭐

Environmental Impacts

There are clear environmental impacts in excessive energy use for the massive computing needs of AI:

  • "[T]he electricity used by AI search for a single second could have been used to power about seven and a half average electric cars." -- from Jalopnik (thanks for the link, Nathan).
  • "In addition to power, these facilities suck up substantial amounts of water to cool their servers, and are often located in places where land is cheap — like deserts. Only a few operators report their water usage, even though a fifth of servers draw water 'from moderately to highly stressed watersheds.' One paper estimates that globally, the demand for water for data centers could be half that of the United Kingdom within the next several years." -- from Jacobin.

Knowledge, Creation, and Data Thievery

The stuff AI is built and trained on is harvested knowledge (and data) of the collective, which was built by people. There's a lot of words to be said about being able to fairly use stuff, and how nobody actually creates in a vacuum, and everything is built off everything else -- all fair.

It just seems different when it's a robot versus a human, and especially AI created/controlled by a corporation whose prime directive is making money for shareholders. I don't buy for even half a second that any of the rhetoric about "advancing human knowledge" is actual motivation for any company. Companies exist to make money, other stated goals are lip service.

The robots' overlords also really don't seem to care that it's thievery. Corporations and machines aren't people and can't participate in social contracts. At face value what he said in that video at the timestamp is correct -- fair use copying, and open sharing is a good summary of the early Internet "vibe check" back in the day. However, it was between people sharing information and content, not corporate entities and their machine pets.

Surveillance Pricing

I found this article a while back. I adore the term "surveillance pricing." Essentially, surveilance pricing is taking the internet ads demographics targeting (e.g., browser used, device used, extensions installed, time spent browsing, etc.) and attempting to transfer it to the real world to make price adjustments for goods and services.

Examples, using things like timing, gait analysis, facial expression analysis, eye tracking, and related data points in a hypothetical grocery store:

  • Did you stop and evaluate prices at every item?
  • Did you pick up an expensive item and then put it back?
  • Did you meander slowly through the store like your time has little value relative to pricing, or did you beeline to your items and throw them into the cart with disregard to price?
  • Are you making specific faces or gestures in response to pricing?

Then compare all that data against similar people exhibiting similar behaviors, plus loyalty card purchase history, plus payment method, plus 1,000 other tiny data points that are all aggregated and turned into a dollar amount adjustment on a baseline price.

This is an area where AI is genuinely terrifying and "useful" -- because it is actually decent at iterating over large datasets, standardizing, and pulling numbers used to draw conclusions. If anything will be a moneymaker, the surveillance pricing thing seems a likely candidate. It has some markers:

  • Correct application of AI in its current form.
  • Business to business for big bucks and less customer support.
  • Not a thing that requires actual intelligence, just generates from data.

It might not be the next household name to consumers, so still not worthy of all the trillions-of-dollars, world-altering hype of AI -- but might become a price tooling backbone for various industries. It's already happening with rents.

All while making everyone's life worse, yay. It's arguable that this stuff can maybe yield a lower price, sure. The way things are currently playing out -- squeezing everyone for every last penny seems like standard operating procedure. Price reduction does not seem a likely future.

What To Do?

Not a clue. My best guess is behavioral and appearance-based deception. AI can't tell when you're being sarcastic, hence the glue pizza and urophagia (or, in memespeak, "Bear Gryllsing").

AI doing real-world "surveillance pricing" can be thrown off by walking and dressing funny, and intentionally adjusting behaviors to counteract AI analysis. "Sleeper" things that don't look or act as expected have always amused and fascinated me. Bonus! Now there's a legitimate and good reason I wear ratty thrifted t-shirts and scuffed old boots.

Intentional garbage goes in, garbage comes out? Pretty much the same idea as "adversarial" image corruption -- like Nightshade, just applied to the real world and people.

This only goes so far. It needs constant refinement as the technology advances, and that is exhausting and not realistic for most people. And it isn't really workable at all for "big deal" stuff -- buying a beater car just to intentionally deceive AI is very, very silly.

We're probably stuck with this bullshit and it's... not great (understatement).

Personal Opinion Dumpster 🗑

On one hand, I know I'm holding myself back and I could move faster, and that I'm missing out on some of the good. It's occasionally tempting to start relying on AI more.

On the other, it's too risky. My math skills (or lack thereof) are a product of overreliance on a machine and shaky fundamental concepts. I don't want to repeat that experience with other important skills such as writing code and making money.

And, stubbornly, it's a fair amount of principle -- who'd have guessed. I don't want to contribute to public knowledge decay, ecological impacts, or copyright thievery. These are not petty hills upon which to die.

Upshots

  • It is cool that newer models remember contexts for your old chats and personalizes further. Although I have qualms already about Google search showing me what it thinks I want based on past data, rather than what I'm actually asking for. So I'm not that convinced on this point.
  • It does help figure out specific words and labels for concepts -- something I often struggle with. E.g., one of the chats I remember with fondness was getting the word "tiering" out of "What's some words or phrases for 'free vs paid'?" (after three rounds of refining the prompt).

Standard-Fare Hallucination Problem

In a few instances I've run into the "hallucination" problem. I don't think any of these instances are all that unique or interesting, but here they are:

  • It told me some of the code I wrote is AI-written. I know I wrote that code, it's just making a best guess based on error-checking, commenting, etc.
  • I fed it a writing sample, and it told me there were grammatical errors. I asked for specifics -- I'm not anywhere near perfect, but I wryte prity gud and catch a lot of my own mistakes, and engage in copious amounts of self-editing. ChatGPT walked my "grammatical errors" back and apologized for making a mistake.

Consensus Expertise Problem

I've run into the "consensus expertise" issue as well. When I asked nebulous questions in the spirit of navigating corporate shark-waters, it provided only standard-fare, corporate-friendly tactics and strategies.

Conversational Reticence (Personal Quirk)

Lastly, and an odd personal quirk, and this is for ChatGPT and "ask-style" AI rather than "autocomplete" style -- I don't like talking and having a conversation to get information, particularly detailed or complex information. I'm not opposed to conversation, it's just typically not my first (or second, or third, or fourth) line of offense. For me, conversation comes after diving in solo and reading a bunch of stuff to try and draw my own conclusions.

I'm pretty sure this is an oddity and not how most people operate -- natural language and conversational flow is probably a very familiar and comfortable option. I get that. I just don't operate like that and a chat-based interface is just not how I'd ideally like to source information.

Links, Articles, Etc.

Even more stuff: