Skip to main contentSkip to page footer

 |  Blog

LLMs and edge computing: efficient data analysis and precise system diagnostics

Data processing and analysis on edge devices is increasing rapidly. Local processing significantly reduces data transmissions, saves bandwidth and opens up new possibilities for system diagnostics using large language models (LLMs).

Data processing and analysis are becoming increasingly important, especially locally on edge devices. This local processing significantly reduces the amount of data transmitted by only forwarding relevant information. This saves bandwidth, relieves the network infrastructure and opens up new possibilities for system diagnostics using large language models (LLMs).

Edge analytics with LLMs enables specialist personnel to identify potential problems and abnormalities directly on-site using voice commands and take action. This contributes significantly to minimizing downtime, especially at locations without direct cable access. In addition, these technologies have adaptive learning capabilities that allow them to continuously adapt to new data and changing conditions. This leads to optimized diagnostics and increased efficiency of maintenance work. 

LLMs are ideal for data analysis as they are based on the processing and interpretation of language. They can recognize patterns in language and generate answers or suggestions based on them. This ability is also transferable to writing and understanding code. In practice, this means that LLMs can understand complex language queries, formulate algorithms and even generate the analysis code directly (see Figure 1). This is particularly useful in areas such as low-level business intelligence, where specific and complex data processing tasks are involved.

Choosing the right LLM is crucial for successful data analysis and should consider criteria such as model size, specific analysis capabilities and processing speed. In addition, prompt engineering plays an important role in improving the quality of the analysis results and exploiting the full potential of the model.

In addition, the way in which data is made available for analysis via a Retrieval Augmented Generation (RAG) system plays an important role (see Figure 2). Retrieval augmented generation (RAG) systems are an essential part of these technologies, combining the capabilities of an LLM with external knowledge bases to improve the generation of accurate answers. A RAG system consists of three main components: the embedding model, which converts text into numerical vectors that represent essential content and meaning; the retriever, which quickly identifies relevant documents that best fit a question based on these vectors; and the LLM, which generates accurate answers based on this information.

The challenge for RAG systems is to select exactly the information that can precisely answer a given question from a large amount of information. Advanced techniques play a decisive role here. This includes improving embedding models, which convert texts into numerical vectors more accurately to enable a more precise match between questions and information. The system's continuously optimized retriever uses machine learning to efficiently identify more relevant information. In addition, the integration of reliable data sources through an efficient data quality management system is of great importance to significantly improve the information base. Finally, the LLM checks and validates the retrieved information to ensure consistency and credibility before generating a final answer.


We offer comprehensive support for the use of language models on edge devices to realize their full potential. Our optimized Retrieval Augmented Generation (RAG) solutions are tailored to your needs and enable effective integration of information into the dialog. This leads to accurate and contextual responses that can significantly increase productivity in various use cases. 

We work closely with you to explore the specific possibilities in your company or for your particular use case to ensure optimal utilization.

About the author


Pascal Scheck is studying computer science with a focus on artificial intelligence. During his internship semester at M&M Software, he had the opportunity to pursue his interest in Large Language Models (LLMs) by evaluating their possible applications on edge devices. This practical experience now feeds into his work as a student trainee in the Data & AI team.

About the author


Rainer Duda is a Data & AI Consultant at M&M Software and supports companies in the development of data-driven business models and the realization of AI-supported applications. For many years, he worked as a data scientist at the renowned Institute for Telematics (TECO) of the Karlsruhe Institute of Technology (KIT) on the Smart Data Solution Center Baden-W├╝rttemberg (SDSC BW) project, among others, and holds lectureships in multivariate statistics and applied data science.

Created by