Industrial LLMs: Automation Apprentices
Automation apprentices powered by large language models enable faster integration of industrial automation capabilities. Companies and system integrators have been integrating ChatGPT into workflows.
Last week’s post on Copilots for Asset Performance, focused on the integration of large language models (LLMs) into asset performance management products primarily found in the Enterprise layer of the Purdue model factory stack. This week we will surface LLMs and Generative AI capabilities applied to products at the Process (Level 0/1) and Control (Level 2) layers.
The Opportunity for AI-assisted Automation Apprentices
Factory automation systems are difficult to design, develop, and commission. Corso Systems, an automation designer and integrator, has developed a six-step framework for an automation project lifecycle that is helpful for understanding where large language models can significantly speed up this process, particularly in Development, Deployment, and Training.
Development is usually the longest portion of any project as it requires the bulk of the effort.
LLMs are currently improving the development phase in two distinct ways. One way is the family of “copilots” popularized by GitHub Copilot which “suggests code completions as developers type and turns natural language prompts into coding suggestions” for robots (e.g. RAPID, TP, INFORM) and CNC (e.g. G-code), etc. The other way is through new ‘foundation models’ designed specifically for industrial applications such as robot picking.
During deployment, we move from testing individual data points and I/O into testing operational systems within the overall process, and then move from individual systems to the process as a whole. Depending on the project this can take on many forms, however, it usually involves running through the process from the beginning, fixing any items needed along the way, validating the fixes, and continuing through the process until the entire process is deployed.
LLMs are currently improving the deployment phase in similar ways to the development phase. Moving to the Level 1/2 layers there are also ‘copilots’ for PLC (Ladder Logic), FactoryTalk, TwinChat, and HMI (Ignition) code generation that orchestrate a process. LLMs analyze system logs and procedures in real-time to identify issues during deployment, providing immediate feedback and potential solutions. LLMs can also quickly generate a wide range of hypothetical test scenarios based on historical data and predefined parameters, aiding in robust system testing.
Consistent and well-led training helps them understand all of the options available to them in those moments, and things like making the documentation available in the HMI exactly when and where they need it can be a huge help on large-scale projects.
LLMs can serve as an intuitive interface for operators to interact with the system, simplifying the process of querying system statuses or troubleshooting issues. LLMs can guide technicians through complex calibration procedures, providing step-by-step instructions and adjusting guidance based on real-time feedback from the system.
Automation Apprentices for Factory Systems
The automation apprentices are roughly ordered from Level 0: industrial robots, sensors, and actuators, Level 1: PLC/DCS, to Level 2/3: operation and control with SCADA, HMI, and WMS.
ABAGY
How ChatGPT Programmed an Industrial Robot didn't stop at the theoretical level – they put the ChatGPT programs to the test on a Yaskawa industrial robot.
Covariant
How is it all connected? ChatGPT, Robotics, and Logistics enhance reliability through generalization, Covariant’s singular and versatile AI model outperforms the more traditional specialized solutions tailored to specific industries, products, or use cases within the warehouse.
AI at its best: A closer look at the Covariant Brain
The Covariant Brain’s foundation model approach involves training one generalized AI on all use cases and item types to achieve the human-level autonomy that we've been missing from the previous generations of picking robots.
Building one AI that works across multiple tasks in the physical world allows our robots to pick the widest variety of SKUs, ranging from nail polish and lemons to soy sauce packets, winter coats, and, even, mixed cases.
This outperforms the more traditional specialized AI models with a narrower scope tailored to specific industries, products, or use cases. The foundation model has proven to be more successful at handling the edge-case scenarios that frequent unstructured real-world fulfillment environments.
Sereact
PickGPT is the first commercially available robotics transformer that enables robots to understand natural language and perceive their environment with an unprecedented level of intelligence and accuracy.
Yaskawa
By utilizing the AI technology “Alliom ” that Yaskawa developed, we can create learning data closer to the real environment on the simulator and pick not only rigid objects but also soft objects with the same hand. Since the AI generation process of (1) generation of learning data, (2) learning, and (3) AI generation can be processed on the simulator, the installation time to actual operation, including AI development, is remarkably shortened, and the accuracy of actual operation can also be improve.
SprutCAM
Éncy AI Assistant provides an AI virtual assistant to help engineers with CNC machining tasks by embracing ChatGPT technologies based on OpenAI API.
CamInstructor
ChatGPT is Evolving! It Now Creates GCODE From Prints!
One of the most exciting developments is the ability to feed ChatGPT an image. This feature addresses a significant challenge I encountered during my previous testing - conveying information about the desired part to ChatGPT. Now, you can simply upload an image or a drawing of your part, making the communication process much smoother.
RoboDK
RoboDK Virtual Assistant is the first step towards a comprehensive generalized assistant for RoboDK. At its core is OpenAI’s GPT3.5-turbo-0613 model. The model is provided with additional context about RoboDK in the form of an indexed database containing the RoboDK website, documentation, forum threads, blog posts, and more. The indexing process is done with LlamaIndex, a specialized data framework designed for this purpose. Thanks to this integration, the Virtual Assistant can swiftly provide valuable technical support to over 75% of user queries on the RoboDK forum, reducing the time spent searching through the website and documentation via manual methods. Users can expect to have an answer to their question in 5 seconds or less.
General ChatGPT (Jakob Sagatowski)
ChatGPT... ChatGPT... It's been on the news every day for the last two months. I've seen it being used for traditional IT-oriented languages/programming, but the question is: can it be used to do PLC-programming? Let's find out!
ABB
LLM-based Control Code Generation using Image Recognition
LLM-based code generation could save significant manual efforts in industrial automation, where control engineers manually produce control logic for sophisticated production processes. Previous attempts in control logic code generation lacked methods to interpret schematic drawings from process engineers. Recent LLMs now combine image recognition, trained domain knowledge, and coding skills. We propose a novel LLM-based code generation method that generates IEC 61131-3 Structure Text control logic source code from Piping-and-Instrumentation Diagrams (P&IDs) using image recognition. We have evaluated the method in three case study with industrial P&IDs and provide first evidence on the feasibility of such a code generation besides experiences on image recognition glitches.
ChatGPT for PLC/DCS Control Logic Generation
Large language models (LLMs) providing generative AI have become popular to support software engineers in creating, summarizing, optimizing, and documenting source code. It is still unknown how LLMs can support control engineers using typical control programming languages in programming tasks. Researchers have explored GitHub CoPilot or DeepMind AlphaCode for source code generation but did not yet tackle control logic programming. The contribution of this paper is an exploratory study, for which we created 100 LLM prompts in 10 representative categories to analyze control logic generation for of PLCs and DCS from natural language. We tested the prompts by generating answers with ChatGPT using the GPT-4 LLM. It generated syntactically correct IEC 61131-3 Structured Text code in many cases and demonstrated useful reasoning skills that could boost control engineer productivity. Our prompt collection is the basis for a more formal LLM benchmark to test and compare such models for control logic generation.
Siemens
Siemens and Microsoft drive industrial productivity with generative artificial intelligence
Siemens’ new Teamcenter app for Microsoft Teams to use AI, boosting productivity and innovation throughout a product lifecycle
Azure OpenAI Service powered assistant can augment the creation, optimization and debugging of code in software for factory automation
Industrial AI to enable visual quality inspection on the shop floor
Siemens and AWS join forces to democratize generative AI in software development
Siemens to integrate Amazon Bedrock into its Mendix low-code development platform to allow customers to create new and upgrade existing applications with the power of generative AI
Beckhoff
TwinCAT projects with AI-assisted engineering
Beckhoff has developed TwinCAT Chat for the TwinCAT XAE engineering environment. With TwinCAT Chat, large language models such as ChatGPT from OpenAI can easily be used to develop a TwinCAT project. This increases productivity in control programming.
Rockwell Automation
The companies are combining technologies to empower the workforce and accelerate time-to-market for customers building industrial automation systems. The first outcome of this collaboration will add Microsoft’s Azure OpenAI Service into FactoryTalk® Design Studio™ to deliver industry-first capabilities accelerating time-to-market for their customers building industrial automation systems.
FactoryTalk Design Studio accelerates with generative AI
“This is still a prototype, so it’s not available to users yet,” explained Gregory. “But we did develop a FactoryTalk Design Studio copilot, which uses the natural-language queries. This is possible because all of ChatGPT sits on large-language models (LLM). We’re now trying to incorporate Rockwell Automation’s domain know-how with them.”
Corso Systems
ChatGPT and Ignition Perspective
We have found a few caveats. First (much like Google) you need to learn how to use the tool. The way you ask questions can significantly impact the results you get. And sometimes you need to feed in information for ChatGPT to know what to do and how to respond. If your request is related to a topic like general web development knowledge which already has a large amount of information available online, ChatGPT can pretty easily write code for you. But if you are working with a something specialized like Ignition—without millions of posts on Stack Overflow—you will need to provide extra information before ChatGPT can give you useful results.
Emerson
Revamp in Action: Using AI to Streamline Your Path to Optimal Control
Take a deep dive into the power of Emerson's REVAMP—a cloud-based advanced software solution and transformative tool that delivers a fully-integrated, digital workflow, using the power of AI, to improve capital efficiency and streamline your path to optimal control.
Zebra Technologies
Zebra Technologies Demonstrates Generative AI on Devices Powered by Qualcomm
Zebra’s TC53/TC58 and TC73/TC78 mobile computers and ET6x Series tablets powered by Qualcomm Technologies – together with Zebra’s asset visibility and intelligent automation solutions – deliver elevated data insight, analysis and recommendations, problem solving, planning and creativity. Front-line workers can utilise a smaller on-device model, even in rural, built-up and underground working environments where connection to the cloud may not be possible. Alternatively, users may switch to a cloud-based app or web browser GenAI tool via Zebra’s Wi-Fi 6/6E and 5G enabled devices.
Cleverdist
We're breaking new ground by harnessing the power of AI and LLM technology, pushing SCADA development into a new exciting era. Discover SmartNodes++ AI assistant, where AI meets innovative software to streamline the construction of industrial WinCC OA control systems like never before.
Create WinCC OA Ctrl++ Test classes and methods in seconds using AI / GTP
Automated testing, often sidelined in industrial solutions due to cost and time, is now virtually free with SmartNodes++. This innovation not only elevates the quality of solutions but also supercharges maintainability, empowering developers to create and innovate fearlessly.
GageList
Harnessing ChatGPT to Develop Calibration Procedures for Test and Measurement Equipment
We wanted to find out how well ChatGPT would perform, so we fed it a prompt to write a detailed procedure for calibrating a Mitutoyo digital caliper. After some experimentation with additional details and instructions to ChatGPT we found the following prompts to provide an excellent first draft.
inVia Robotics
inVia Robotics Wins “2023 Top Software & Tech” Award for its AI Technology
inVia’s warehouse automation software leverages AI that transforms our customers’ operational data to generate a daily order fulfillment plan that orchestrates all labor, inventory, and equipment. The plan is continuously re-generated and updated as warehouse conditions change (and they often do).
This system is our equivalent of an LLM (large language model), which we refer to as our LWM (large warehouse model). It’s an intelligent transformer that allows the system to make all of the decisions that have to be made as plans change. It scales decision-making beyond what a warehouse manager could possibly do.
