CIO OPINION
Moreover , easy integration with existing workflow makes this the least-disruptive approach .
However , a strong dependence on the application provider ’ s security and data protection controls , can lead to security and data privacy risks for organisations . There are also chances that applications with embedded AI may not be able to deeply understand the context of a conversation or task , leading to less-accurate or less-relevant responses .
Embed generative AI APIs
Enterprises can build their own applications , integrating generative AI via foundation models APIs . Most closed-source generative AI models , GPT-3 , GPT-4 , PaLM 2 , etc . are available for deployment via cloud APIs . This approach can be further refined by prompt engineering . In prompt engineering , the underlying foundation model is frozen — this provides the ability to use the same model across a variety of use cases . a new dataset to incorporate additional domain knowledge or improve performance on specific tasks . This often results in custom models that are dedicated to the organisation . This approach can result in improved performance and reduce hallucinations as the models are fine-tuned with organisational data and , or domain-specific data for particular tasks .
But foundation models fine-tuned for specific use cases might lose their ability to be extended to broader use cases . Moreover , the cost of using a fine-tuned model , inference cost can be significant , even if the cost of fine-tuning training is not high .
Build custom foundation models
Organisations could ultimately build their own foundation models from scratch , fully customising them to their own data and business domains . If adequate data governance is in place , then the organisation will have complete control over the training datasets and model parameters .
Additionally , foundation models can perform new tasks with adequate accuracy with a limited number of highquality samples . This approach has its own benefits , but prompt engineering is a nascent field , where best practices are only emerging , and for which new skills are required .
Extend generative AI models
Retrieval augmented generation , RAG enables enterprises to retrieve data from outside a foundation model , often your internal data and augment the prompts by adding the relevant retrieved data . This will improve the accuracy and quality of model response for domain-specific tasks . Extending the models via a RAG approach can provide an appropriate balance between bringing organisational context into foundation models without the complexity and cost of modifying the underlying models .
This can significantly increase use-case alignment and reduce bias . This approach grants greater control to the organisations over the model , however , the cost of training and maintaining a large , generative AI model can be exceedingly high .
Most organisations can deploy some , if not all , of the approaches described above , depending on the use case , technical knowledge , maturity of the organisation , and time-to-market requirements .
When comparing deployment approaches and choosing the one that delivers business value , IT leaders need to be aware of various important factors – total cost of ownership , TCO , integration of organisational and domain knowledge , implementation complexity , model accuracy performance and ability to control security and privacy . p
However , implementing a RAG approach involves redesigning the technical architecture and workflow to include new technology components . The knowledge about these technology components and the overall architecture is pretty rudimentary in most enterprises . These additional components also carry additional costs .
Extend via fine-tuning
Fine-tuning takes a large , pretrained foundation model as a starting point and further trains it on a new dataset to incorporate additional domain knowledge or improve performance on specific tasks .
Fine-tuning takes a large , pretrained foundation model as a starting point and further trains it on
www . intelligentcio . com INTELLIGENTCIO AFRICA 39