Closed Loop LLM

(Examples: GPT-4 (RLHF), Claude, Gemini, RAG systems)
In a Closed Loop LLM, the model’s self-learning capability is developed and refined through human interaction and usage. This approach is akin to learning through mistakes and corrections by experience. The model adapts and improves based on the feedback it receives from users, which can lead to more accurate and contextually appropriate responses over time.

Characteristics of Closed Loop LLM:

Usage of Closed Loop Models:

• Chatbots with User Feedback Mechanisms: Customer service chatbots incorporate mechanisms for users to provide feedback on the responses they receive, which is then used to improve the model
• Virtual Assistants Learning from User Interactions: Virtual assistants like Siri, Alexa, and Google Assistant learn and improve from the interactions they have with users, adapting to individual preferences and common queries

Closed Loop LLM Examples:

1. Google Assistant’s Personalisation:

• Description: Google Assistant learns from user interactions to personalise responses. For instance, if a user frequently asks about weather forecasts for a specific city, Google Assistant will adapt to provide quicker, more direct answers for that location.
• Demonstrates: Adaptive learning and human-in-the-loop feedback, reducing the need for users to input the same queries repeatedly

2. Chatbots in Customer Service:

• Description: Many companies use chatbots on their websites that learn from customer interactions. For example, if a user frequently inquiries about return policies, the chatbot will improve its responses over time to better address such queries.
• Demonstrates: Human-in-the-loop learning and the potential for reduced bias through diverse customer feedback.

3. Spotify’s Recommendation Algorithm:

Description: Spotify uses a form of Closed Loop LLM to recommend music based on user listening habits. The more a user listens to certain genres or artists, the more tailored the recommendations become.
• Demonstrates: Adaptive learning and the ability to mitigate bias by learning from individual user preferences

Open Loop LLM (Examples: LLaMA 2/3, BERT, RoBERTa, BLOOM, Falcon)
In contrast, an Open Loop LLM relies on its training dataset for its learning and development, with minimal to no adaptation based on user interactions post-deployment. This method can increase the chances of bias and wrongful interpretations if the training dataset is not carefully curated and diverse.

Characteristics of Open Loop LLM:

• Static Learning: The model’s learning is primarily based on its initial training dataset, with little to no adaptation to new information or changing contexts post-deployment
• Risk of Bias and Errors: If the training dataset contains biases or inaccuracies, the model may propagate these issues in its responses, potentially leading to harmful or misleading outputs
• Efficiency in Controlled Environments: Open Loop LLMs can be highly effective in controlled or specific domain applications where the scope of queries is limited and well-defined

Usage of Open Loop Models:

• Pre-Trained Language Models Without Fine-Tuning: Models like the original versions of BERT and RoBERTa, which were pre-trained on large datasets but not fine-tuned with specific user interaction data, fall into this category.
• Domain-Specific Models Trained on Static Datasets: Models trained for specific tasks, such as medical diagnosis or legal document analysis, based on static datasets without ongoing user feedback loops

Open Loop LLM Examples:
1. BERT for Sentiment Analysis:

• Description: BERT (Bidirectional Encoder Representations from Transformers) was initially trained on a large corpus of text and then applied to sentiment analysis tasks without further training on user interactions. Its performance is based on the patterns learned from its initial training dataset.
• Demonstrates: Static learning and the potential risk of bias if the training dataset does not represent diverse viewpoints.

2. Domain-Specific Medical Diagnosis Models:

• Description: Some AI models are trained on static datasets of medical records and literature to diagnose diseases. These models do not learn from new patient interactions post-deployment.
• Demonstrates: Efficiency in controlled environments and the risk of propagating biases or inaccuracies present in the training data.

3. Legal Document Analysis Tools:

• Description: AI tools used in legal firms to analyse documents are often trained on large, static datasets of legal texts and precedents. They do not adapt to new legal documents or user feedback post-deployment.
• Demonstrates: Efficiency in specific domain applications and the potential for errors if the training dataset is not comprehensive or up-to-date

Conclusion: By distinguishing between Closed loop and Open loop LLMs, developers and organisations can make informed decisions about which approach best suits their specific applications. This choice rests on several critical factors, including the need for adaptability to new trends and user behaviours, the potential for bias in model outputs, and the value of integrating user feedback for continuous improvement. As the landscape of language model deployment continues to evolve, understanding these distinctions will be pivotal in harnessing the full potential of LLMs while mitigating their limitations

Overcoming Impediments

In continuation of the AI blog series, this blog gives viewpoints on the impediments for AI adoption from PoC to production. When we take a look at the current AI adoption, we see that the chip makers are way ahead of their software counterparts, which in my view is a first we have seen over the past few decades!

What’s holding the adoption back? Is it proper use case fitment or non-availability of base data or vulnerability of current security frameworks to DDOS (Distributed Denial of Service) or the CSOs thinking that current systems or processes can’t handle protection policy (data, network)? Well, one can’t be sure! It can be a combination of all these points! And, this concoction is the reason why adoption of AI use cases from PoC to Production is pegged at a maximum of a measly 28% conversion rate.

Diving into the primary obstacles-

Primary Obstacles

1. Data Quality 

Inadequate base data quality hinder AI model effectiveness.
At least 30% of generative AI (GenAI) projects will be abandoned after proof of concept by the end of 2025, due to poor data quality, inadequate risk controls, escalating costs or unclear business value, according to Gartner, Inc. According to the IBM Big Data & Analytics Hub, poor data costs the US economy $3.1 trillion every year. Research from Cognilytica points out that over 80% of the time in most AI and Machine Learning projects is spent on Data preparation and engineering tasks. This emphasises the need for high quality data obtained by leveraging the solutions and best practices in data preparation, management and engineering for implementing effective AI algorithms

2. Risk Controls 

Inadequate risk management frameworks and controls.

According to a recent survey report by Deloitte, tracking generative AI adoption and challenges, risk and governance are the among the top 4 major challenges companies have in implementing generative AI applications and tools. Only 23% of the companies were highly prepared for ri
k and governance

3. Infrastructure Costs:  

Prohibitive costs of infrastructure development and maintenance.
In a recent Gartner Data & Analytics Summit in Sydney, an analyst cited the cost of projects as a big pressure on generative ai deployment, with upfront investments ranging from 5 million to 20 million

4. Uncertain ROI:  

Difficulty quantifying financial benefits of AI use cases.

As AI applications vary widely across industries, it is difficult to establish standardized benchmarks for expected ROI. Typically, around 90% of AI initiatives do not meet their ROI targets, primarily due to challenges in deploying models efficiently and aligning outcomes with business goals

5. Technical Expertise 

Shortage of skilled professionals to create and maintain AI models.

41% of businesses are struggling to find employees to support their generative AI initiatives, according to Enterprise Strategy Group’s recent survey

6. Trust Trust of the models that are used, where the quality & source of data is not known
7. Model Scalability and Performance:

According to a 2022 study by MLOps Community, 62% data scientists face issues with model performance when moving to production. PoCs are typically conducted in a controlled environment with limited resources. The AI model needs to be scaled to be moved into production. A robust deployment strategy needs to be in place to scale and serve concurrent users without degradation in performance

That being said, there are areas where AI adoption is seeing greater traction, there are a few use cases that are into production successfully 

Successful AI Adoption Use Cases

• Productivity Measurement

Retail, hospitality and customer service industries leverage AI to measure process and employee productivity.  According to the Forbes Advisor survey, 53% of businesses apply AI to improve production processes, while 51% adopt AI for process automation and 52% utilize it for search engine optimization tasks such as keyword research

• Data Analysis

AI-driven data segmentation and ROI calculations are gaining traction.

• Automation Agents

Self-sustaining routines automate tasks efficiently. 75% of users want to leverage AI to automate workplace tasks

• Code assistance

Aid in software development. An Enterprise Strategy Group survey of application developers found that 63% used generative AI in production, citing faster code creation and improved customer support as top benefits.  Business executives observe 55% developer productivity improvement due to generative AI adoption

Challenges

To bridge the gap between PoC and production, it’s crucial to address the following challenges:

• Data Quality Improvement

Ensuring reliable, high-quality data by defining a data governance framework to ensure accuracy, completeness, consistency, and timeliness of the data used in AI models

• Risk Management Frameworks

Developing and implementing robust risk controls

• Cost-Effective Infrastructure
  

Exploring cloud-based, scalable solutions. Platforms like AWS, Google Cloud Platform (GCP), and Microsoft Azure offer specialized AI and machine learning services that allow businesses to scale resources up or down based on demand. Applications that require real time processing such as Self-driving cars, wearable devices, and smart home appliances can leverage edge AI which enables models to be deployed closer to where the data is generated, reducing data transfer costs and reliance on centralized cloud processing

• Clear ROI Definitio
  

Establishing measurable financial benefits. A study from ESI ThoughtLab and Deloitte found that organizations with mature AI strategies generally see higher ROI as these companies focus on robust data management, effective tracking, Privacy, security, ethics and strong alignment with strategic goals from the onset

• Skill Development

Investing in AI talent acquisition and training. Businesses need to focus on upskilling, knowledge sharing, and fostering a culture of continuous learning among existing employees by investing in the right AI training programs for data engineers, developers and analysts

Embracing AI governance after tackling these impediments, organizations can unlock the full potential of AI, driving successful adoption from proof of concept to production. However, there are concerns emerging as AI adoption is set to increase multifold

Emerging Concerns

• Data Governance

AI output, data publication and usage raise concerns around copyright laws, data infringement, liability, and monitoring

• Regulatory Compliance

EU and UK AI liability and copyright laws require adherence, necessitating AI governance and monitoring standards

In my next blog, let’s look into the governance and regulatory boxes. Until then… 

————————————————————-———————————————————————–

References: 

https://www.recodesolutions.com/why-enterprises-struggle-to-move-gen-ai-led-automation-beyond-pilot-projects-and-into-real-applications/

https://www.linkedin.com/pulse/from-prototype-production-overcoming-ai-deployment-hurdles-xzr1c/

https://www.techtarget.com/searchenterpriseai/feature/Survey-Enterprise-generative-AI-adoption-ramped-up-in-2024

https://fair.rackspace.com/insights/eight-blockers-transitioning-ai-production/