4 levels of using AI solutions in software development

At this level, we are discussing using AI tools to enhance developer productivity. The software being developed can be anything and does not necessarily include AI-based functionality. Since the result of programming is code, which is a specific type of text, it makes sense to speed up this part of the production with AI tools. This is where AI development services come into play, offering tools that streamline the coding process.

Certain coding parts are very creative, and this is where the greatest added value is created. These are the design phases when the structure of the developed system is planned. These phases are not worth automating.

However, certain coding parts are repetitive and not particularly creative, often just "intelligent copy-paste." Examples include generating boilerplate code, testing, refactoring, and documentation. Large language model-based generative AI is best suited for these semi-creative text-centric tasks, and AI tools can be confidently used here. These are called code assistants, and there are quite a few available on the market, such as Github Copilot, Tabnine, and Jetbrains AI Assistant. Their application should be understood in the context of AI software development.

The short answer is increased productivity, which can be 20-50-100% according to various studies. “According to LogiNet’s experience, this largely depends on the nature of the task. For intelligent copy-paste tasks, 100% seems believable, but generally, it is more in the 20-30% range,” highlighted Zsigmond Máriás, the CEO of LogiNet Systems Ltd.

However, there are a few significant dangers to consider, and I would highlight two of the most important ones.

The coding assistant helps the developer do what they do faster. Consequently, if they are doing something incorrectly, they will do it faster and more productively, which can have unforeseen consequences.

Another problem is that the code completions generated by these tools can be incorrect. It is very dangerous to accept the suggestion without understanding it, and it is even worse if we do not understand what has been generated. In the hands of juniors, this is especially dangerous both for the project and for the developer, so it is not advisable to leave this process to junior developers.

There are many other factors and opportunities, such as open-source assistants, but the essence is that using these tools properly can save time safely, and no special AI knowledge is required for their use.

At the second level of using AI, we integrate AI into the product itself. Here we are talking about using someone else's AI solutions via APIs for well-defined sub-tasks in the development of a digital product, such as a mobile app or web service.

This approach is worth using if our product is complete and viable without AI, we need quick results, we do not have significant AI expertise, and we do not expect the system to learn and improve over time. AI consulting services can guide this process, ensuring the integration is seamless and effective.

API services democratize the development of AI applications by simplifying entry. This is beneficial, but we should not expect that by connecting a couple of APIs and prompting, we will create an application that consistently solves a complex problem. If we do, the results are easily reproducible.

We need to understand that what we can achieve with this is limited. Perhaps most importantly, in this case, the knowledge in the system is what the tool manufacturer embedded during training. Our knowledge is present only in a limited way, extending to the prompt, the size of which is increasing in newer products.

This level requires general development knowledge and prompt engineering experience, based on common sense, systematic trial and error, and familiarization with the tool used. In most cases, this is what is meant by AI development.

The most important is the platform risk: the terms of use and characteristics of the models and services we use can change to our disadvantage. Another issue is legal risks: we may inadvertently use copyrighted materials, and our data or our clients' data may be used as training data for the services we utilize.

When we hit the limitations of the previous level, where a single API and prompt engineering are no longer sufficient, we need to move to the next level. AI development companies can be instrumental in creating a comprehensive AI stack.

Typically, the goal at this level is to significantly enhance an already viable digital product by integrating, fine-tuning, and creating a collaborative environment of multiple AI products and technologies. Here, our objective might include embedding significant specific knowledge into our system, as well as incorporating the possibility for feedback, learning, and improvement. In essence, we are creating a custom and collaborative AI stack.

This means that in practice, we do not rely on just one or two tools; rather, we use multiple tools whose cooperation yields the desired result. We use AI for certain tasks and control them with other AI tools and deterministic algorithms. The tools are fine-tuned, embedding significant unique knowledge specific to the problem. Often, the system is provided with memory using vector databases, classical databases, and document stores. It is also crucial to use ready-made products, technologies, models, and APIs that allow for the necessary customization.

Typical tools at this level include various programming libraries like Langchain, Llamaindex, AutoGPT, and usually the Python programming language, vector and other databases (e.g., Pinecone, PG-Vector, Elasticsearch), open-source language models (e.g., Llama 2 and it's further trained variants), MistralAI models (as of early 2024), and all APIs and technologies from the previous level (ChatGPT and similar). There are many available models and applications with fine-tuning options, providing a wide range of choices.

This approach is worth considering when dealing with more complex, multi-step workflows that handle multiple use cases exceed the limitations of APIs and cannot be consistently resolved with prompting alone, particularly when feedback is involved in the workflow.

We can confidently call this level AI development, which can set us apart in terms of the final result from those who only incorporate some external tool. The result is much harder to replicate. However, do not expect this level to solve new, unresolved problems, as this will likely require proprietary models and the next level of development.

At this level, real knowledge about the operation of AI tools is required, not just their use. We must know how to further fine-tune, parameterize, and operate the models, and the developers involved must understand the background as well. Typically, BSc/MSc professionals with a background in computer science education are needed.

When we need to develop a product for tasks where parameterizing and integrating existing tools is no longer sufficient, and we must solve problems for which there are no ready-made solutions, we need to create new models or further train base models beyond fine-tuning.

At this level, we are usually developing a product that would not be viable without AI solutions. We solve a problem with custom AI models and tools, resulting in a significantly better outcome. The key is to create unique models, which requires a precise understanding of how to train AI tools and how to operate such tools. At this level, the issue of training data becomes prominent, as we will need a lot of data to train and validate the models.

We will still need the entire toolkit from the previous levels, as creating custom models is not an end but a means. We should apply ready-made solutions where available. Typically, we start tackling the problem with Level 3 tools, but during development, we identify issues requiring our model.

Here's a LogiNet client example: In a robocall tool, we need to determine if the person we are talking to has finished speaking and is waiting for a response. Excellent voice activity detection (VAD) models and algorithms are available for this purpose, which can tell if someone is currently speaking. However, they cannot determine if the person has finished their statement or is just pausing. This is a problem requiring a custom model.

Typical tools at this level include machine learning (ML) methods (e.g., neural networks, transformers, reinforcement learning) and data science methods for generating training data. We use low-level ML libraries (e.g., TensorFlow, PyTorch, OpenCV), and control tools for parallel execution environments (e.g., CUDA), and knowledge of Python and C++ is essential.

It is crucial to have specialized AI/ML DevOps for running training and operating models, as training and operation at this level are non-trivial.

Development at this level requires computer science professionals specializing in AI or ML development. We typically expect this knowledge from university-level computer science MSc or PhD graduates.

Utilizing AI tools in software development is inevitable. If not in the actual coding process, then at least in the production phase, AI tools must be incorporated. Leverage AI consulting services to identify the best tools for your needs and ensure you are maximizing the potential of these technologies.

Avoid applying AI solutions to every problem just because it is trendy. Some issues simply do not have effective AI-based solutions. For example, using ChatGPT for pricing decisions is not advisable. Instead, focus on areas where AI business solutions can truly add value and improve efficiency.

Be aware of which AI solutions are suitable for which problems and understand the level they operate at. This knowledge helps set realistic expectations for the outcomes and identifies what can and cannot be achieved. Whether you are engaging in AI development, using APIs, or developing a full AI stack, knowing the right approach is crucial.

Match the appropriate level of AI integration with the right tools and expertise, whether it is an internal team or an external supplier. Ensuring the chosen tools and experience align with your objectives is crucial for successful implementation. Consider working with an AI development company or AI consulting firm to access the expertise you need. For instance, contact centre AI solutions can be particularly effective for improving customer service processes