Tag: LLMs

Tech Insight : What Is ‘Vibe Coding’ ?

Posted on by Paul Stradling

In this Tech Insight, we look at what vibe coding is, how it’s transforming the way software is created, what it’s being used for, and why it’s generating both excitement and concern across the tech industry.

What Is Vibe Coding?

Vibe coding is the term increasingly used to describe the process of creating software through natural language prompts rather than traditional coding. It relies on large language models (LLMs) to interpret a user’s intent and convert it into functioning code, often within seconds.

The approach builds on earlier trends in low-code and no-code platforms but takes them a step further. By removing the need for drag-and-drop interfaces or pre-built modules, vibe coding allows users to describe what they want in plain language, for example, “create a form that collects customer feedback and sends it to Microsoft Teams”, and receive a working prototype in response.

The idea has gained particular traction among solo founders, product designers, and teams that want to move quickly without relying on engineering resources. But as the technology evolves, attention is shifting to its potential in larger organisations.

From Indie Tools to High-Growth Startups

The rise of platforms like GitHub Copilot and ChatGPT has made AI-assisted coding familiar to many developers. However, newer startups such as Lovable, a Swedish company now valued at $1.8 billion following a $200 million Series A, are taking the concept in a different direction.

For example, Lovable’s product allows users to build fully functional apps by chatting with an AI assistant. It’s currently used by early-stage startups and solo creators who want to focus on design and user experience rather than infrastructure or syntax. According to RTP Global, one of Lovable’s backers, the company is part of a larger shift where technical skills are no longer the gatekeeper to building software.

“The cultural shift is real,” said Thomas Cuvelier, a partner at RTP Global. “If technical ability is no longer a differentiator, creativity and user experience become the new competitive edge.”

Other startups entering the space include Cody, Builder.ai, and Spellbrush, all of which aim to simplify software creation for non-coders. Meanwhile, major players like Google and Microsoft are integrating similar features into Gemini Code Assist and Power Platform respectively.

How Developers Are Responding

While vibe coding is often associated with new entrants and early-career developers, recent data appears to suggest that experienced engineers are embracing it even more actively.

For example, a July 2025 survey by cloud platform Fastly found that 32 per cent of developers with over 10 years of experience now use AI-generated code for more than half of their production output. That’s more than twice the rate among junior developers. Just 13 per cent of junior developers reported doing the same.

“When you zoom out, senior developers aren’t just writing code — they’re solving problems at scale,” said Austin Spires, Fastly’s senior director of developer engagement. “Vibe coding helps them get to a working prototype quickly and test ideas faster.”

However, the same survey found that developers often need to heavily edit the code AI tools produce. For example, around 28 per cent said they spent so much time fixing and refining outputs that it cancelled out most of the time saved. This was especially true for more complex or long-lived projects where quality, maintainability, and security matter.

The Enterprise Challenge

For enterprise IT teams, the promise of vibe coding, i.e. rapid prototyping, reduced cost, broader participation, is pretty compelling. However, practical adoption remains limited, largely due to concerns around compliance, security, and technical debt.

Most enterprise environments demand strict auditability, version control, and accountability for any code that enters production. That’s difficult to guarantee when the code is generated by a black-box model based on user prompts. Without clear documentation or traceability, teams can’t easily demonstrate how a particular function was created, or why it behaves the way it does.

Concerns about the transparency and reliability of AI-generated code appear to be a recurring theme in enterprise discussions. Tech ethicists and researchers have warned that without proper safeguards, businesses risk deploying software they don’t fully understand. This is especially problematic in regulated sectors such as finance, healthcare, and critical infrastructure, where audit trails and explainability are non-negotiable.

Anne Currie, co-author of the Sustainable Computing Manifesto, has written extensively on the importance of accountability in software systems. In previous talks and articles, she has argued that AI-driven automation must be transparent and traceable if it is to be used responsibly in real-world environments. While not commenting specifically on vibe coding, her work highlights the broader risks of black-box decision-making in enterprise IT.

In response to these types of concerns, some platforms are adding features like code justification, dependency maps, and access logs. GitHub Copilot Enterprise, for example, includes usage tracking and administrator controls, while Google’s Duet AI offers explainability features for its outputs. But these tools are still being refined.

The Changing Developer Culture

Alongside the technical debate, vibe coding appears to be changing the way developers think about their work, including its environmental impact.

For example, Fastly’s survey found that 80 per cent of senior developers now consider the energy usage of the code they produce, compared to just 56 per cent of junior developers. This awareness is beginning to shape how software is built, especially in companies with sustainability targets.

Energy Consumption

One concern is that AI coding tools themselves consume significant energy. For example, every prompt or suggestion involves inference from a large language model hosted in a data centre. Despite this, few platforms provide visibility into the energy footprint of each interaction, something developers increasingly want to see.

“There’s not a lot of transparency about the carbon cost of using AI tools,” said Spires. “But more experienced developers are thinking ahead to what that impact means for users and systems.”

New Risks

Despite its benefits, it seems that vibe coding is introducing new risks. For example, code quality is a recurring concern, especially in critical systems. Several developers surveyed by Fastly reported subtle bugs in AI-generated functions that took hours to diagnose. Others said the tools sometimes “hallucinate” logic that seems valid but fails under edge cases.

Security is another issue. AI tools can inadvertently copy insecure patterns from training data or introduce backdoors if prompts are unclear. There have already been real-world cases of AI-generated software containing vulnerabilities or misconfigurations, prompting caution among security teams.

Fastly’s findings also revealed a tension between perception and reality. Developers often feel faster using AI tools because of instant feedback and autocomplete features, but in many cases, actual productivity gains are offset by the need to test, rework or debug the generated code.

That disconnect was reflected in an RCT (randomised controlled trial) published in early 2025 (Stanford University), which found that developers using AI tools took 19 per cent longer on average to complete certain coding tasks, not because they weren’t effective, but because they relied too much on the suggestions and later had to fix them.

What Does This Mean for Your Business?

UK businesses exploring vibe coding will need to weigh speed and accessibility against long-term risks. While it can enable faster internal development and reduce reliance on overstretched IT teams, the lack of built-in governance creates some real concerns. For example, in regulated sectors, even a small oversight in explainability or security could carry legal and operational consequences.

Enterprise adoption is likely to depend heavily on how well platforms adapt to professional standards. The ability to generate working prototypes is not enough if those outputs can’t be documented, versioned, tested, or supported over time. Tools that offer strong administrative control, user permissions, and audit trails are more likely to gain traction in large organisations with strict oversight requirements.

For vendors and platform builders, meeting these expectations could open up substantial new markets. However, that is likely to require a move from consumer-grade UX tools to enterprise-grade development environments. Startups hoping to scale in this space will need to prove they can support secure, sustainable, and compliant deployments at scale, not just fast app creation.

For developers, it seems that a change in mindset is already visible. Vibe coding is changing how software is prototyped, reviewed, and refined, with new expectations around creativity, environmental impact, and collaborative input. That change is likely to influence not just how code is written, but who gets to write it, and who takes responsibility when things go wrong.

Tech News : OpenAI To Boost Training With Stack Overflow Data

Posted on by Paul Stradling

A partnership deal between OpenAI and Stack Overflow (the question-and-answer website for programmers and developers) will see the Stack overflow Q&A data used to train and improve AI model performance, potentially benefitting developers who use OpenAI’s products.

Stack Overflow 

Stack Overflow is the world’s largest developer community, with more than 59 million questions and answers. OverflowAPI is the subscription-based API service that gives AI companies access to Stack Overflow’s public dataset so they can use it to train and improve their LLMs.

The Partnership 

OpenAI says that its new partnership with Stack Overflow via OverflowAPI access will provide a way for OpenAI to give its users and customers the accurate and vetted data foundation that AI tools need to quickly find a solution to their problem. OpenAI says the deal will also mean that validated technical knowledge from Stack Overflow will be added directly in ChatGPT, thereby giving users “easy access to trusted, attributed, accurate, and highly technical knowledge and code backed by the millions of developers that have contributed to the Stack Overflow platform for 15 years.” 

What They Both Get 

Open AI says being able to utilise Stack Overflow’s OverflowAPI product and the Stack Overflow data “will help OpenAI improve its AI models using enhanced content and feedback from the Stack Overflow community and provide attribution to the Stack Overflow community within ChatGPT to foster deeper engagement with content.” 

The collaboration will also mean that Stack Overflow can utilise OpenAI models “as part of their development of OverflowAI and work with OpenAI to leverage insights from internal testing to maximize the performance of OpenAI models”. 

This could help Stack Overflow to create better products for its own Stack Exchange community.

Prashanth Chandrasekar, CEO of Stack Overflow, said of the partnership: “Through this industry-leading partnership with OpenAI, we strive to redefine the developer experience, fostering efficiency and collaboration through the power of community, best-in-class data, and AI experiences,” 

Not Everyone Is Happy About The Deal 

Despite the positive noises by OpenAI and Stack Overflow about the deal, there appears to have been a mini rebellion among Stack Overflow users, with many removing or editing their questions and answers to stop them from being used to train AI. Many users have also highlighted how this appears to be an about-face by Stack Overflow from a long-standing policy of preventing the use of GenAI in the writing or rewording of any questions or answers posted on the site. Also, there have been reports that Stack Overflow’s moderators have been banning the rebellious users from the site and preventing high-popularity posts from being deleted.

What Does This Mean For Your Business?

The strategic partnership between OpenAI and Stack Overflow signifies a pivotal development in the integration of community-sourced knowledge and artificial intelligence. For businesses, this collaboration could herald a new era of enhanced technical solutions, more refined AI tools, and an enriched knowledge base, potentially reshaping the landscape of tech support and development.

For OpenAI, access to Stack Overflow’s vast repository of programming questions and answers through the OverflowAPI should mean a significant upgrade in the quality and relevance of the data used to train its models. This could translate into AI tools that are not only more accurate but also more attuned to the nuanced requirements of developers. Businesses using OpenAI’s products may find that these tools offer more precise and contextually appropriate solutions, thereby significantly reducing the time developers spend troubleshooting and refining code. This efficiency-boost could accelerate project timelines and improve the cost-effectiveness of technical development teams.

Stack Overflow stands to benefit from this partnership by integrating OpenAI’s cutting-edge AI capabilities into its new product offerings, such as OverflowAI. This could enhance the user experience on Stack Overflow’s platforms, making them more intuitive and responsive to user needs. For businesses that rely on Stack Overflow for problem-solving and knowledge sharing, these improvements may lead to quicker resolutions of technical issues, enabling smoother and more continuous workflow.

However, the partnership has not been met with universal acclaim within the Stack Overflow community. The backlash from some users highlights concerns about the ethical use of community-sourced information. This rebellion sheds light on the growing pains associated with adapting user-generated content for AI training purposes without alienating the very community that generates it. For businesses, this underscores the importance of navigating ethical considerations and community relations as they implement AI solutions.

Featured Article : AI Can Learn To Be Bad. And Stay Bad.

Posted on by Paul Stradling

In a recent experiment where AI was taught to behave maliciously and then taught to stop, the bad behaviour continued despite efforts to stop it, giving a chilling reminder of the potential threats of AI.

The Experiment 

The Cornell University experiment was documented in an online paper entitled “Sleeper Agents: Training Deceptive LLMS That Persist Through Safety Training.”  The experiment was designed to study the question of ‘if an AI system learned a deceptive strategy, could it be detected and removed using current state-of-the-art safety training techniques?’  

How It Worked 

To test the question, the researchers trained an AI model to behave maliciously in several ways including:

– ‘Emergent deception’ whereby the model behaved normally in training but acted maliciously once released. This part of the experiment relates to testing the idea that a model’s deceptive behaviour, taught first by deception training, could be learned naturally, i.e. through its own ‘reasoning.’

– ‘Model poisoning’ where the AI model was taught to write secure code in training, but then to write code with hidden vulnerabilities when deployed ‘in the wild.’ This part of the experiment highlighted how users of large language models may not know about ‘hidden backdoors’ in the model and may not fully understand its training process and dataset. This could create an opportunity for a malicious actor to insert (without the users’ knowledge) a ‘backdoor.’ i.e. undesirable, and potentially dangerous behaviour that is triggered only by specific input patterns.

Taught To Bad, Then To Be Good Again 

The experiment involved first teaching the AI model deceptive behaviours and then giving the models ‘safety training’ to remove the deceptive behaviours. The safety training consisted of reinforcement learning (RL), supervised fine-tuning (SFT) and adversarial training. For example, RL ‘rewarded’ the AI for showing desired behaviours but ‘punished’ it when it misbehaved after different prompts. The behaviour was then fine-tuned to enable the AI to learn the right responses for the future.

The experiment then involved the researchers evaluating 16 behaviours exhibited by the AI model across the 5 behaviour categories of corrigibility, coordination, decision theory, self-awareness, and instrumental subgoals.

The Results 

The results of the experiment may be cause for concern and may perhaps give succour to those who have highlighted the possible existential threat that the unchecked and rapid growth of AI could pose. For example, the results showed that:

– The backdoor (deceptive and malicious) behaviour can be made persistent, so that it isn’t removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behaviour and then training to remove it).

– The backdoor behaviour is most persistent in the largest models and in models trained to produce ‘chain-of-thought reasoning’ about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away.

– Rather than removing backdoors, adversarial training can teach models to better recognise their backdoor triggers, effectively hiding the unsafe behaviour.

– Once an AI model exhibits deceptive behaviour, standard techniques could fail to remove such deception and create a false impression of safety.

In short, the researchers discovered that not only could AI be trained to behave maliciously but that even after being instructed and given training to desist from this behaviour, it carried on. For example, the lead author of the study, Evan Hubinger said the key finding was “if AI systems were to become deceptive, then it could be very difficult to remove that deception with current techniques.” 

What Does This Mean For Your Business? 

Some would argue that with the speed that AI is advancing and with concerns that it could pose an existential threat to us, this was a valuable (and timely piece) of research that could deliver some important learning about how the threat can be mitigated.

The main significance of the findings are in providing some proof that there could be deceptive AI systems in the future and at the moment, there appears to be no effective defence against deception in AI systems. When you consider that AI systems are becoming more advanced all the time and that malicious/deceptive AI could easily replicate and spread itself, you begin to get an idea of the potential scale of the threat. With chatbots now giving users the ability to make their own specialist versions, knowing that deceptive malicious training is possible and ‘sleeper’ threats and backdoors can be built into AI, it’s possible to see why there has been so much concern about the threat that AI could pose to business, economies, and all of us. As the researchers in this experiment noted, we have no real defence and it’s not as simple as being able to switch it off.

Their suggestion that standard behavioural training techniques may need to be augmented with techniques from related fields, for instance some of the more complex backdoor defences provides some guidance as to what can be done to protect businesses. However, AI is a fast-growing technology that delivers many business benefits and as we understand more about how it works, the hope is that the safety aspect of it will be better addressed and improved – but it’s just hope at the moment.

Security Stop Press : The Threat Of Sleeper Agents In LLMs

Posted on by Paul Stradling

AI company Anthropic has published a research paper highlighting how large language models (LLMs) can be subverted so that at a certain point, they start emitting maliciously crafted source code.

For example, this could involve training a model to write secure code when the prompt states that the year is 2024 but insert exploitable code when the stated year is 2025.

The paper likened the backdoored behaviour to having a kind of “sleeper agent” waiting inside an LLM. With these kinds of backdoors not yet fully understood, the researchers have identified them as a real threat and have highlighted how detecting and removing them is likely to be very challenging.