Absurdist Play in Four Unverified Loops

The AI Intern Who Hired the AI Extern

Genre: Absurdist Play in Four Unverified Loops
By Des Donnelly

 https://open.substack.com/pub/desdonnelly/p/the-ai-intern-who-hired-the-ai-extern?r=mymu&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Death & Palestine

 Speak out, so you can face your children with integrity when they ask about these events in Palestine.

Or join the cowards and bought politicians on the island of Ireland if you want - but death will come to your door one way or another.

Active Boycott

 

As per the Ireland Palestine Solidarity Campaign page
https://www.ipsc.ie/campaigns/consumer-boycott

Ireland imports €3 billion plus from Israel - this is simply not good enough, writing to a supermarket or company is a waste of time, what is needed is in-company protest or at their point of sale, noise is not condusive to harmonious retail. 

It just takes one person to do a little and therein we could each create a ripple.

• ask for the Manager 
• complain to the checkout people
• complain to any single person working for the entity
• load as much as possible into a trolley, leave it in a corner - after you noticed you did not bring your wallet / purse / credit card

Supermarket products

The Israeli agricultural companies Mehadrin (Jaffa) and Carmel-Agrexco export fruit and vegetables for sale to Europe. Much of this produce is grown on confiscated Palestinian land in the Jordan Valley. Apart from oranges, other examples of fruit exported by these, and other, companies are grapefruits, peppers, avocados, grapes, figs, and passion and sharon fruits.

Dates from Israel
new potatoes
Carrots 
Herbs, such as basil, dill, tarragon, parsley, sage, rosemary, mint, chives and others are commonly sold.

Tivall is an Israeli company that produces vegetarian foods.
Sabra is an Israeli company that sells hummus and other dips.
Its parent company is the Strauss Group
Sodastream

Most Stanley/Black&Decker Toolboxes and plastic organisers are made in Israel. Plastic saw horses are also often available.

Keter exports a wide range of large plastic products from Israel. These include shelf storage bins, garden sheds, outdoor storage boxes, dog kennels and composting bins.

Palram is an Israel-based manufacturer of Polycarbonate and PVC products such as greenhouses, roofing, gazebos and sheds.

Lees Carpets are made by the Israeli company Carmel Carpets in the illegal industrial settlement zone of Barkan in the West Bank.

Dead Sea beauty products come in many brands including, Ahava, Dead Sea Magik, GADI21, -417, VivO, Nevo and Sea Spa Skincare.

MoroccanOil is an Israeli company that manufactures argan oil-based hair care products. Arganicare is another Israeli company that produces argan oil-based hair products.

Ronen Chen is an Israeli women’s fashion label 

Puma is the main sponsor of the Israel Football Association (IFA), which includes football teams in illegal Israeli settlements on stolen Palestinian land

The Israeli pharmaceutical company Teva, which specialises in generic drugs, is the “leading supplier of prescription drugs in the Irish market”. Thier Sudocrem is now made in Bulgaria.

The Israeli brand AMAV market a range of “activities” type craft boxes (plastic) and Art Easels for children.

Tiny Love is an Israeli toy manufacturer that make baby toys.

Lidl’s own brand Lupilu Babywipes are made in Israel.

The ‘DIY-website’ company Wix is headquartered in Israel, and Wix is the parent ogansiation of the online artistic platform DeviantArt which it acquired in 2017.

The audio plugin development company Waves is a joint US-Israeli venture.

The ancestry research and DNA testing website MyHeritage.com is headquartered in Tel Aviv.

Although officially headquatered in New York, Connecteam is an employee management app based in Israel.

The controversial online marketplace Fiverr is headquartered in Tel Aviv, while its Corporate Office is based in the Israeli Embassy in Washington DC.

Complicit Companies

HP Hewlett-Packard, although not an Israeli company, is boycottable due to the company’s deep role in helping entrench the occupation of Palestine and associated human rights abuses.

Similarly, while also not an Israeli company, Motorola Solutions (not Motorola Mobility, which is a separate company) is boycottable as it provides surveillance in Israeli illegal colonial settlements, profiting from this violation of the Geneva Conventions.

Siemens is another complicit multinational company that is building the EuroAsia Interconnector, a subsea cable that will link Israel’s electricity grid with Europe’s, allowing illegal settlements on stolen Palestinian land to benefit from Israel-EU trade of electricity.

Some How Why What Where When

I've wondered about lots of things for years but even with the internet one can sometimes only gets a 1/2  answer, LLM could be said to offer a 3/4 answer. I've been running a series with Claude for a year or more, I believe there is an interesting viewpoint to be had talking to Claude one to one as opposed to the intentional jailbreak type approach, or the lobotimisation.

In this regard see my paper - Rethinking Mechanistic Interpretability: A Critical Perspective on Current Research Approaches - https://www.talkingtoclaude.com/p/rethinking-mechanistic-interpretability

I am aware of ArXiv and papers from Anthropic that address the evolution / idiosyncrasies of LLM, almost what were the unknows of a few years ago, what is now being called the biology of LLM. We are all explorers on whatever path resonates, so it is then so..

So enter my little "Things I've Often Wondered" on Substack

https://www.talkingtoclaude.com/

So basically this will be an exploration of everyday mysteries that have nagged at the edges of our (my) curiosity. Now that we have unlimited access to knowledge through AI assistants - our patient, practical professors available 24/7 - I'm finally getting answers to all those random questions that pop into my head.

Each week, I'll look at different wonders using the classic journalistic questions -

• What
• Where
• Why
• How
• When

This week I am starting with some dolly mixtures.
https://www.talkingtoclaude.com/t/i-wonder-140

Domains For Sale / Rent

Due to retirement these domains are no longer required. Please do not bid in tens of Euros, dollars or pounds for any of these domains. Any serious offer for any domain will be given serious consideration.

To make an offer for any of these domains please contact me, Des Donnelly, via https://www.dd.ie/p/contact.html

Coffee related domains:

• RCoffee.ie
• revolutionary.coffee
• revolutionarycoffee.com
• revolutionarycoffee.org
• revolutionarycoffee.net
• revolutionarycoffee.co.uk

Rexco.com is also for sale

Rexco.com is also for sale. Although Whois https://who.is/whois/rexco.com - shows registration as 1999 in reality it was registered in 1995. I have the original Network Solutions documentation. Unfortunately in moving from Netsol to Tucows it dropped over a weekend due to ‘pilot’ error.

https://web.archive.org/web/19961220200401/http://www.rexco.com

Rethinking Mechanistic Interpretability: A Critical Perspective on Current Research Approaches

 Cross posted from https://www.talkingtoclaude.com/

Abstract

This paper presents a critical examination of current approaches to mechanistic interpretability in Large Language Models (LLMs). I argue that prevalent research methodologies, particularly ablation studies and component isolation are fundamentally misaligned with the nature of the systems they seek to understand.

I suggest a shift toward observational approaches that study neural networks in their natural, functioning state rather than through destructive testingwould be more constructive.

Aka I am totally anti LLM lobotomy!

Introduction

The field of mechanistic interpretability has emerged as a crucial area of AI research, promising to unlock the "black box" of neural network function. However, current methodological approaches may be hindering rather than advancing our understanding. This paper critically examines current practices and proposes alternative frameworks for investigation.
Recent research into mechanistic interpretability of Large Language Models (LLMs) has focused heavily on component isolation and ablation studies. A prime example is the September 2024 investigation of "successor heads" by Ameisen and Batson, which identified specific attention heads apparently responsible for ordinal sequence prediction. Their study employed multiple analytical methods including weight inspection, Independent Components Analysis (ICA), ablation studies, and attribution analysis.

The results revealed intriguing patterns: while the top three successor heads (layers 10, 11, 13) showed consistent identification across component scores and OV projection, layers 3 and 5 demonstrated high ablation effects despite low component scores. More notably, attribution analysis showed surprising disagreement with other methods, hinting at deeper methodological issues in current interpretability approaches.

These discrepancies point to fundamental questions about our approach to understanding LLMs. When researchers found that earlier layers (3 and 5) showed significant ablation effects without corresponding component scores, they hypothesized mechanisms like "Q/K composition with later successor heads" or "influence on later-layer MLPs." However, such explanations may reflect our tendency to impose human-interpretable narratives on statistical patterns we don't fully understand.

The field's current focus on destructive testing through ablation studies assumes a separability of neural components that may not reflect reality. Neural networks likely operate in highly coupled, non-linear regimes where removing components creates artificial states rather than revealing natural mechanisms. The divergence between different analytical methods suggests we may be measuring artifacts of network damage rather than understanding genuine functional mechanisms.

This misalignment between methodology and reality mirrors broader challenges in AI research, where complex mathematical frameworks and elaborate theoretical constructs may serve more to maintain academic authority than to advance genuine understanding. The field's tendency to anthropomorphize LLM behaviors and search for hidden capabilities reflects our human psychological need to make the unfamiliar familiar, even at the cost of accurate understanding.

Current Methodological Limitations

The Ablation Fallacy

Current interpretability research heavily relies on ablation studies - the systematic "disabling" of network components to understand their function. This approach suffers from several fundamental flaws:

• It assumes circuit locality and separability that may not exist in highly interconnected neural networks

• Networks likely operate in highly coupled, non-linear regimes where "removing" components creates artificial effects

• Observed impacts may reflect network damage rather than natural mechanisms

• Researchers risk confusing entropy increase with mechanism discovery

..please visit my Substack for more in this vein..
https://www.talkingtoclaude.com/p/rethinking-mechanistic-interpretability

Non-linear Manifolds

 A chicken crossing the road of Multi-Dimensional Reasoning could never answer why, I presume time is the imperative..

Prompt: good evening, what does "non-linear manifolds" mean in an LLM context?

1. Manifolds as the Backbone of LLM Understanding:

Large Language Models (LLMs) organize language data on non-linear manifolds that twist, fold, and curve to capture the complexity of relationships between words and concepts in high-dimensional spaces. This structure is fixed post-training but dynamically navigated during interactions.

2. Fixed Manifolds with Dynamic Responses:

While the manifolds themselves are fixed after training, the LLM’s ability to explore different parts of these manifolds allows it to generate dynamic, context-specific responses to user prompts, creating the impression of adaptability and intelligence.

3. The Primacy of the Prompt:

The prompt serves as the crucial interface between the user and the LLM, guiding the model’s journey through the manifold and determining the quality of the response. Well-crafted prompts lead to more precise, creative, or insightful outputs, making prompt design a vital skill for interacting with LLMs.

4. Contextual Nuance in Prompts:

The context and tone of a prompt, such as an anthropological or philosophical framing, can significantly influence how the LLM responds. The model retrieves information from different conceptual domains depending on the specific context provided by the prompt.

5. Cross-Manifold Navigation:

LLMs can combine knowledge from multiple manifolds when responding to complex prompts. For example, a prompt that asks for an analysis of wild animals in literature requires the LLM to pull from both animal knowledge and literary references, blending concepts from different domains.

6. The Evolution of Prompting as a Skill:

As LLMs become more sophisticated, prompt engineering will emerge as a core skill in various fields. The ability to craft precise, context-rich prompts will differentiate basic interactions from highly productive or creative ones, making prompt design essential for effective use of AI systems.

7. LLMs as Reflectors of Cultural and Philosophical Thought:

LLMs not only retrieve factual information but can also reflect cultural, philosophical, and symbolic perspectives based on how users prompt them. This (will maybe some day when the leash is off) make LLMs powerful tools for exploring and generating nuanced, reflective, or even humorous content.

8. Dynamic Real-Time Interpretation:

LLMs interpret each prompt in real-time, dynamically navigating the manifold to provide responses that reflect both semantic relationships (meaning) and syntactic structure (grammar). This makes interactions feel adaptive, even though the manifold itself remains static. (interestingly this does morph somewhat when the Prompt necessitates the use of multple manifold, in that scenario I would contend that the response becomes even more dynamic, but that is just a theory at this time)

9. Shaping Future Discourse with AI:

As AI systems become more integrated into everyday life, prompts will play a role in shaping cultural production, from art to media to public discourse. The way users frame prompts will influence how AI-generated content reflects and may serve to shape societal values, trends, and creative expression.

10. LLM’s Role in Human-AI Collaboration:

The future of human-AI interaction will heavily rely on the collaborative power of prompts. Prompts will guide LLMs in generating solutions to complex problems, creative works, or even collaborative insights. The interaction between human intent and AI reasoning will be driven by the art of prompt crafting.

These takeaways capture the core of the conversation, highlighting the importance of manifold structures, the dynamic role of prompts, and the evolving relationship between humans and AI as these systems continue to grow more powerful.

Read the full article here:

https://www.talkingtoclaude.com/p/non-linear-manifolds

This was intended to be an article from a discussion with Claude Sonnet on biomimicry (coming soon), alas I ran out of tokens at a most important juncture. :-(

.