EN 
SV
PT-BR
ES
DE
Technical writers have always been gatekeepers of producing a single source of truth. They coordinate with Subject Matter Experts in any organization to produce documentation. Technical writer meticulously listens to stakeholders such as product managers, developers, UX designers, customer support, customer success, sales, and marketing team conversations and documents them. Technical writers then draft the articles providing absolute clarity in the content for better comprehension. If any ambiguous content is being discussed, quickly intervene to remove ambiguity. A technical writer produces many artifacts such as user manuals, standard operating procedures, troubleshooting guides, process manuals, software “how-to” guides, etc. These artifacts have absolute truth in them because of which “trust” becomes inherent. Some of the artifacts are used for compliance and regulatory filling of products in the hardware domain for manufacturing and medical devices. Thus, technical writer’s workflows are critical in ensuring the right and accurate information is present in those user manuals. The quality control workflows ensure that technical writers’ work is peer-reviewed and vetted by Subject Matter Experts before publication. The end users of the documentation always relied on a PDF or online knowledge base for accessing accurate information.
Current Trend in Search
The search engine which is based on “lexical keywords” brings in relevant articles based on user keywords. Thus, the end users always go to the article source to get the information they need.
Example of lexical search
However, things have changed after the introduction of GenAI technology such as ChatGPT, Gemini, Claude, and so on. Now, GenAI-powered search engines are taking over the semantic search.
Example of semantic search
The GenAI-based agents act as an interface between end users and documentation. Given the generative nature of this AI technology, it is harder to have control over what is generated as a response to user questions! This impacts “trust”!
Also Read: What are the responsibilities of technical writers in the GenAI era?
Limitations of Current GenAI
The characteristic of Artificial Intelligence (AI) is that it is non-deterministic! It is inherent in the technology implementation. The GenAI tools are based on the Large Language Models which are built to predict the next token (token is ¾ of a word).
The GenAI tools generate a response based on user questions which are often called “prompts”. The Retrieval Augmented Generation (RAG) architecture is used to supplement the knowledge gaps present inherently in the LLMs. The RAG approach can provide better context and content that is up to date with facts to help LLM generate a better response to the input prompt. The quality of the generated response depends on
- Quality of the prompt
- Quality of the content
- Underlying LLM
Quality of the prompt
The quality of the prompt determines the quality of the generated response. If the prompt is vague and contains ambiguous terms, it might confuse the RAG architecture when retrieving article chunks that might have ambiguous information, leading to a non-factual response that is not grounded in truth.
Quality of the content
The quality of the content determines the accuracy of the generated response. If the content is poorly written, it makes it harder for the LLM to correlate prompts to the context and generate a response. The content structure should include scope/purpose, prerequisites, how-to, FAQs, and so on so that LLMs can better infer the context for the entered prompt. All the nuances about the topic should be covered in detail inside the article content.
Underlying LLM
The underlying LLM that is used to generate a response plays a huge role in ensuring that the generated response is supported with trustworthy content. The LLMs such as ChatGPT-4, Claude, and Llama 3 are powerful and can produce more accurate responses. If the advanced model is not used, the LLMs may suffer from hallucination problems leading to generating less accurate or non-factual responses. There is a cost factor involved in choosing an advanced LLM for your use cases.
Hallucination in GenAI
Hallucination is when the GenAI-generated response is not factual. The below charts show that GenAI must generate response the right response for the prompts when the answer is completely or not available inside the content. When the GenAI tool is making up facts confidently which is deviating from the ground truth, then the GenAI is said to be “hallucinating”. It is harder to eliminate the hallucination as it is inherent in the LLM itself. To a certain extent, the researchers have solved the bottom quadrant problem to ensure that GenAI produces accurate responses based on content availability.
Role of technical writers in trust
The bigger problem for technical writers is that now GenAI acts as an interface between their knowledge base and the end users. Technical writers do not have control over GenAI to ensure trust. Given the probabilistic nature of the GenAI tools, it takes more work to produce a deterministic response. Even a slight change in prompts changes the generated response significantly. The bigger question is how to ensure that GenAI tools can trusted. There are a few ways and emerging best practices to achieve this. They are
- Focusing on content quality and ensuring no ambiguous content is present in the knowledge base. A well-curated business glossary can be fed into the GenAI tools to understand user prompts better resulting in an accurate response
- A list of source/citation articles that are used to generate a response can be shown to the end users so that they can verify the generated response
- Utilizing advanced LLM by leading GenAI provider
Technical writers need to evaluate GenAI tools by preparing a lot of test cases (questions along with ground truth). They can then compare the generated responses with ground truth to compute metrics on reliability, correctness, and context relevancy. This helps to build trust factor into putting GenAI tools in production.
Also Read: Leveraging AI Prompts for Technical Writing
Closing remarks
The GenAI research is accelerating, and it is just a matter of time before hallucinations disappear. The LLMs are getting more powerful and versatile with long context windows. The LLMs are getting cheaper so this technology will become ubiquitous in many products we use. Building trust in the generated response utilizing good UX principles and better content should be the primary objective of technical writers who can produce more GenAI-friendly content.
An intuitive AI-powered knowledge base software to easily add your content and integrate it with any application. Give Document360 a try!
GET STARTED
–