Conf42 Large Language Models (LLMs) 2024 - Online

- premiere 5PM GMT

Content preview

The CFP is still open, and the content is subject to change
Speaker Talk

Karin Wolok

Using Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg with Stock Data and LLM

In this talk, we will discuss how to use Apache NiFi, Apache Kafka, RisingWave, and Apache Iceberg to process and analyze stock data. We will start by discussing the different components of the stack and how they work together. We will then show how to use this stack to ingest, process, and analyze stock data. Finally, we will show how to use an LLM to generate predictions from the analyzed data.

Andrey Cheptsov

Leveraging open-source LLMs for production

The talk will cover: 1. Pros and cons of open-source LLMs vs OpenAI and alike 2. Basic economics of hosting open-source LLMs 3. Open-source serving frameworks 4. State of cloud GPU 5. An overview of main open-source LLMs

Aarushi Kansal

Large Language Models and Observability

Large Language Models (LLMs) like GPT-4 have profoundly impacted the AI landscape, powering applications from customer service to automated content creation. However, these powerful models also come with their own set of challenges, including issues of output inconsistency and model drift. In traditional engineering, observability plays a crucial role in maintaining system reliability. How can we apply these principles to LLMs to ensure they are both effective and safe in production environments? In this talk, we delve into the unique challenges that LLMs present, such as addressing issues related to output drift and model brittleness. We will explore a variety of tools and techniques...

Ben Fistein

The future of search

Open-source LLMs and vector databases allowed us to build a search technology that allows users to describe their search terms using common language, returning all results that fit the category of this language. For example, we can now search an entire e-commerce store with a query such as "a slightly bitter beverage people drink to wake them up" and our tool should return results related to coffee. In fact, the user can use any input language or even images as their query.

Eugene Neelou

Governance, Risk, and Compliance for Large Language Models

The talk explains how the explosion of Generative AI introduces new risks for organizations that use tools like ChatGPT in the workplace. It's essential to manage risks to avoid cyber incidents, private data leaks, regulatory fines, or copyright infringement. The presentation covers relevant governance, risks, and compliance controls for security, privacy, and legal professionals. The examples include asset management and observability for LLMs, privacy for data protection and compliance, security against cyber risks, and safety for responsible AI adoption.

Dan Savino

Safe AI Implementation Strategies: Protecting Your Company's Future While Embracing Innovation

"Safe AI Implementation Strategies: Protecting Your Company's Future While Embracing Innovation" is a talk designed for executives who are interested in leveraging artificial intelligence (AI) to drive innovation and growth in their organizations. The talk focuses on the importance of implementing AI in a safe and responsible manner, considering the potential risks and challenges associated with this rapidly evolving technology. During the talk, the following key points will be addressed: - Understanding the potential benefits and risks of AI: Executives will learn about the various ways AI can benefit their organizations, such as optimizing and automating business processes, providing...

Jason Tan

Large Language Models is What We Need to Create Jarvis from Ironman

Prepare to embark on a captivating journey as Jason takes the stage to unveil the incredible world of Large Language Models (LLMs) and their pivotal role in bringing science fiction to life. In this thought-provoking talk, you will delve deep into the groundbreaking advancements in LLMs that are redefining the very fabric of software development. Join us as we explore the complexities and opportunities of crafting an AI assistant akin to the legendary Jarvis from Iron Man. Jason will unravel the following key themes in this enthralling session: - Advancements in LLMs: Discover how LLMs are pushing the boundaries of what was once considered science fiction and turning it into reality. -...

Spiros Economakis

Leveraging Large Language Models in a Communication Platform

In the evolving landscape of technology, understanding the depth of context within communication platforms is crucial. With the advent of GenAI, we aim to explore how this technology can significantly enhance context awareness and productivity. Our session delves into the transformative power of GenAI, spotlighting its ability to streamline intricate processes such as Site Reliability Engineering, Change Management, Production Readiness Reviews, and Incident Management. Moreover, GenAI addresses the pervasive issue of FOMO (Fear Of Missing Out) while prioritizing overall well-being, fostering a balanced work environment. In this presentation, we don't shy away from the critical...

Aldan Creo

How to use ChatGPT without getting caught

Join us as we explore the world of detecting AI-generated text, such as what you get from ChatGPT. Have you ever wondered how experts uncover it? We'll spill the beans! We'll start from scratch, briefly explaining how the generation of text using AI works. No previous background in the field is required! Then, we'll discuss classifier systems for detection, black and white box models, watermarking, and manual techniques for detection — cool stuff to keep in your toolbox! But here's the twist—these systems are not foolproof. There are ways to get around them, and we'll explore how. Ready to uncover the mysteries? See you there!

Sam Naji

Navigating the New Internet: LLMs as the Key to Web4 Transformation

This talk delves into the transformative role of Large Language Models (LLMs) in shaping the future of the internet, particularly in the Web4 landscape. We'll examine the impact of LLMs on user interface design and environmental sustainability, using real-world applications like the Kizana AI fashion advisor as case studies. Insights will be drawn on GPU demand, the environmental footprint of LLMs, and the potential of Small Language Models (SLMs) for specialized tasks. Additionally, we'll explore the ethical considerations in data handling and privacy in the Web4 context, highlighting the synergy between LLMs and SLMs for an optimal user experience.

Ankit Virmani

How to develop framework for Real-Time RAG in LLM Applications

This talk will go over a real-life example/demo of how to keep the vector DB up to date using streaming pipelines, importance of RAGs and how they can be used to eliminate hallucinations, which can have a huge catastrophic impact on the outputs of LLMs. It would be a great session for data and machine learning engineers would want to learn through a deep dive session on how to fine tune LLMs using open source libraries.

Deepak Karunanidhi

Future of LLM's and Machine learning Productionization

In this talk, we'll dive into the captivating realm of the Future of Large Language Models (LLMs) and the intricate process of Machine Learning Productionization. I will unravel the latest breakthroughs in language models, shedding light on their evolving capabilities and the transformative impact on various industries. Moreover, we will explore the art and science of deploying machine learning models into real-world applications, ensuring scalability, efficiency, and tangible business impact. From language understanding to production integration, this session promises an enlightening glimpse into the exciting future of AI.

Chloé Caron

How well do LLMs detect anomalies in your data?

Data quality issues can significantly impact revenue, with a reported average 12% loss for US companies (Experian report). Addressing this starts with the identification of anomalies. This talk follows a journey into building an anomaly detector, iteratively expanding on the solution to boost its accuracy. In this talk, we will begin with a simple anomaly detector using OpenAI’s API. We will then delve down multiple exploratory tracks including prompt engineering and the impact of input data types. To wrap things up, we will undertake a comparative analysis of tools, including BigQuery and Mistral, to highlight their unique...

Luca Bianchi

Unleash GenAI while remaining cost-effective with serverless

GenerativeAI is rapidly reshaping the tech landscape. Its implementation, especially in intricate scenarios, often involves a sophisticated dance among application code, users, and multiple Large Language Models (LLMs). The complexity of coordinating these elements extends beyond the capabilities of basic API calls. While some vendors offer means to access LLMs, they fall short in guiding developers to construct an LLM framework that effectively balances responsiveness, accuracy, and cost-efficiency. Consequently, many projects grapple with unforeseen costs, pushing them beyond budget. Addressing this challenge, our proposal centers on a serverless architecture. This approach adeptly...

Bongani Shongwe

Running an open source LLM

Large language models (LLMs) like GPT-3 have demonstrated impressive capabilities in natural language processing. These models can generate human-like text and power applications like chatbots, content generators, and code assistants. OpenAI, Bard, Bedrock are just a few services which have arisen in order to provide generative AI services. But different use cases might prevent you from using these ready to serve services, or you may have actually considered stepping away from these services and running your LLM model. However, these models are computationally intensive and require specialised hardware to deploy. In this talk, we will explore the options available for harnessing LLMs,...

Samuel Baruffi

Vectoring Into The Future: AWS Empowered RAG Systems for LLMs

The power of Retrieval Augmented Generation (RAG) for LLMs is transformative, and AWS tools can help unlock this potential. This talk explores how to use vector databases, custom silicon with Inferentia, and LLMs on SageMaker and BedRock to augment your RAG systems. Learn how to maintain up-to-date reference documents, execute relevancy searches, and deliver accurate responses, all while optimizing cost and performance. Let's pioneer the future of AI together!

Bobur Umurzokov

LLM for better developer learning of your product

Developing a tech product is not just about coding and deployment. It's about the learning journey that goes into building and utilizing it as well. If you have a developer-oriented product, it is about ensuring that developers understand your product at a deep level through the docs, tutorials, and how-to guides, improving both their own skills and the quality of the work they produce. Nowadays AI can not only generate docs from code but also it makes easy to find specific information or answer questions about your product using a chatbot for a better developer experience. It is life-changing for these project docs maintainers. This talk explores how LLMs(Large Language Models) and LLM...

Manuel Heinkel

Generative AI Security — A Practical Guide to Securing Your AI Application

The pace of innovation in generative AI offers immense opportunities while also introducing new security challenges. Those include new threat vectors, explainability, governance, transparency, and privacy for large language models. As organizations seek to leverage generative AI for innovation, security leaders must take concrete steps to enable rapid experimentation without compromising security. We will begin our talk by understanding the scope of generative AI applications, based on their intended use and the potential risks associated with their deployment. We will then discuss key strategies for securing generative AI applications, including threat modeling, guardrails,...

Zain Hasan

Building Chatbots with Advanced Retrieval-Augmented Generation Techniques

Chatbots are becoming increasingly popular for interacting with users, providing information, entertainment, and assistance. However, building chatbots that can handle diverse and complex user queries is still a challenging task. One of the main difficulties is finding relevant and reliable information from large and noisy data sources. In this talk, I will present some of the latest advances in retrieval-augmented generation(RAG) techniques, which combine the strengths of both retrieval-based and generative approaches for chatbot development. Retrieval-based methods can leverage existing text documents to provide informative and coherent responses, while generative methods can produce...

Joshua Arvin Lat

Building our own LLM Vulnerability Scanner to audit and secure AI applications

Over the next few years, we'll see more organizations building various AI-powered tools and systems. While most AI-powered tools can be built using 3rd-party services and APIs, we'll see more companies using their own LLMs and hosting it in their own private network environments. For one thing, having a self-hosted LLM would guarantee greater control over data privacy and security. In addition to this, companies would gain the much needed flexibility when customizing their LLMs to specific business needs and constraints. At this point, most professionals are not aware of the security threats and potential security vulnerabilities when building AI-powered applications utilizing self-hosted...

Filipp Shcherbanich

Applying LLMs in real PHP projects: Transform Your Code with Advanced AI

In the speech "Applying LLMs in Real PHP Projects: Transform Your Code with Advanced AI," we explore the revolutionary impact of Large Language Models on PHP development. This talk dives into how LLMs, like GPT and BERT, are reshaping the PHP landscape, offering new possibilities for code generation, optimization, and problem-solving. We will discuss the practical aspects of integrating these sophisticated AI models into real-world PHP projects, highlighting both the opportunities and challenges that come with this cutting-edge technology. Attendees will learn about various use cases where LLMs can enhance PHP development, from automating routine coding tasks to providing advanced code...

Marcin Szymaniuk

AI chats — what nobody told you: the conundrums of business integration.

ChatGTP's growth in popularity is unmatched. Yet, only some companies are integrating it into their systems to make something more sophisticated than a single prompt. One of the reasons is that integration requires both specific knowledge and effort. However, it is not the only one. When one considers integrating their production system with a ChatGPT-like tool, the privacy of your data and the cost it will generate are two essential matters to consider. Careful analysis of what data one sends, where it's being processed, and how much and when one will have to pay for it are the questions to be addressed. These questions are crucial for defining a business case which has the potential to...

Santosh Nikhil Kumar

Advanced API Design for Scalable and Fault-Tolerant Data-Intensive Distributed Systems

This proposal delves into RESTful principles, asynchronous operations, security considerations, techniques for optimizing data retrieval through pagination and filtering, exploration of caching and rate limiting, importance of idempotent operations and error handling in maintaining data consistency and facilitating seamless integration with evolving systems. Learning objectives: This topic aims to provide practical insights and guidelines for modern software engineers, architects, and developers involved in the creation and evolution of APIs for data-intensive distributed systems. By adopting the proposed advanced API design principles, organizations can achieve a balance between...

Timothy Spann

Adding Generative AI to Real-Time Streaming Pipelines

In this talk I walk through various use cases where bringing real-time data to LLM solves some interesting problems. In one case we use NiFi to provide a live chat between a person in Slack and several LLM models all orchestrated via NiFi and Kafka. In another case NiFi ingests live travel data and feeds it to HuggingFace and WatsonX.AI LLM models for summarization. I also do live Q&A from hosted chat webpages. We also augment LLM prompts and results with live data streams.

Newsletter partner

Media partners

Awesome tech events for

Priority access to all content

Video hallway track

Community chat

Exclusive promotions and giveaways