Conference Agenda

Brief Description and Outline:

This tutorial sheds the light on the small and locally deployed large language models (LLMs) and vision language models (VLMs). It starts by deploying them on our local machines with the help of Ollama. This is followed by the implementation of API endpoints – using Python – that utilize the VLM for image captioning via zero-shot prompting. Finally, the tutorial concludes with an evaluation of the AI-generated captions using the dedicated metrics, such as BLEU and ROUGE

Goals:

Familiarize the audience with the importance of locally deployed LLM or VLM models, especially for implementing services that don’t require API keys or the disclosure of organizational data outside its network

Presenters Experience:

Dr. Osama Hamed is a postdoctoral researcher specializing in applied artificial intelligence and machine learning. He has several years of experience working in the fields of natural language processing (NLP) and AI. Dr. Hamed earned his PhD in NLP from the University of Duisburg–Essen in 2019. Additionally, he accumulated several years of teaching experience as both an assistant professor and while holding a master’s degree.

Target Audience:

Researchers who are interested in local LLMs, Computer Vision or NLP.

Basic to good understanding of Gen AI, LLMs as well as text generation evaluation metrics.

Participants should bring their own laptops.

Keywords:

Natural Language Processing, Computer Vision, Image Captioning, LLMs, Vision-Language Models (VLMs).