EN 
Large Language Model providers are publishing system cards to bring transparency into their model behaviors. Only 20% of the top 100 model cards on Hugging Face fully incorporate all recommended sections, according to a large-scale analysis of 32,111 model cards. As regulations like the EU AI Act push frontier labs to publish a system card for every major release, that documentation gap is becoming a compliance and trust problem, not just a style one.
Regulations such as the EU AI Act mandate the publication of reports on what AI models could do, where they fail, and what data shaped them. Many LLM providers such as OpenAI, Anthropic, Thinking Machines and so on are publishing system cards for their frontier models. System cards and datasheets for documentation are technical documentation. It is the responsibility of a technical writer to speak with researchers to document the characteristics of frontier models, how they are trained, and the behavior they exhibit. Given the importance of the system card for each frontier model release, technical writers need to curate a list of questions they need to ask to collect information to document the system card. Given the technical nature of the system card, it is important to write content in layperson’s terms so that it can benefit a wide range of readers rather than the technical community.
Unlike traditional API documentation or product documentation, system cards document how an AI system behaves rather than how it is used. Their purpose is to explain model capabilities, limitations, evaluation methodology, safety controls, and intended use so that developers, researchers, regulators, and enterprise buyers can make informed decisions. As organizations adopt frontier models more rapidly, system cards are becoming both a transparency document and a critical governance artifact.
What Are System Cards and Datasheets?
A system card is a detailed document that describes a trained machine learning model, and a datasheet documents the dataset behind it. Both were introduced to formalise how AI is reported, and both feed the same goal: letting a reader decide, quickly, whether to trust and how to use a given model.
Although the terms are sometimes used interchangeably, system cards, model cards, and datasheets serve different purposes within AI documentation. A model card focuses on an individual trained model, a datasheet documents the characteristics of the underlying dataset, while a system card describes the behavior, safety evaluations, operational constraints, and deployment considerations of the complete AI system. Together, these documents provide traceability across the AI lifecycle, from data collection to model deployment.
Components of System cards and datasheets
While a system card and a datasheet serve the same underlying goal, they document different things. A system card focuses on the trained model itself, its purpose, behavior, and performance, while a datasheet focuses on the data that shaped it. Together, the two documents give a reader a complete picture: what the model does, and what it was built on. The sections below break down what each document typically contains.
A system card document contains:
- Purpose of the model
- Performance and capabilities
- Limitation of model
- Benchmarking results
- Ethical considerations and caveats
- Training and evaluations
A datasheet for datasets contains information about
- Data sources and their properties
- Data characteristics
- Nuances in datasets
- Data provenance
- Dataset copyright
Model cards were introduced in “Model Cards for Model Reporting” (Mitchell et al., 2019), and datasheets in “Datasheets for Datasets” (Gebru et al., 2018). System cards are suited for end users deciding to trust a frontier model, developers integrating a model into software applications, and engineers using it for agentic use cases. This system card artifact complements data cards, API references, and model cards for frontier models.
Find out how Document360 helps technical writers document complex AI systems, faster!
Book a Demo
Scope of System card
A system card must cover enough ground that every major model behaviour is documented, every benchmark is reported, and every known limitation is stated plainly. That scope typically breaks into two groups: the sections every system card needs, and the sections that round it out.
Core Documentation Areas
- Model identity and scope: This helps to orient the readers and get a holistic perspective of the frontier model
- Training data and process: This provides provenance and accountability for what data the model is trained on and the nuances behind data collection
- Capabilities and performance: This covers their capabilities around SWE, reasoning, long context, agentic search, multimodal, and multilingual. Some report also goes into vertical-specific capabilities in life sciences, health, cyber, and so on
- Content safety and harmlessness: This covers guardrails and their measured limits
- Robustness and adversarial: This focuses on security postures that are critical for agentic use cases. This covers agentic safety mechanisms such as prompt injection, malicious use of agents, and so on
- Hallucination and factuality: This covers honesty and factuality to quantify reliability of generated responses
- Alignment, honesty, misuse propensity: This centers around alignment assessment of the model with respect to the model provider’s constitution
- External and third-party testing: This includes reports from independent research bodies such as METR, Apollo, SecureBio, and Irregular
Additional Sections
Many system cards also cover threat modelling, safety training, real-time safeguards, red-teaming efforts, and security controls. These sections vary more by provider, so it’s worth checking a few recent system cards before deciding what to include.
The Technical Writer’s Role in System Card Documentation
Given the technical nature of the system card, it is important for a technical writer to understand technical jargon used in Large Language Model research, development, build, and deployment. In terms of skill set, a technical writer needs to understand the landscape of :
- Graphical Processing Unit (GPU): Having concrete ideas about GPU architecture and the software stack on GPU (e.g CUDA)
- LLM pre-training: Understanding Transformer architecture, encoder/decoder, tokenization, and so on
- Text embedding, prompt engineering, and topic modeling
- LLM post-training
This helps technical writers to ask the right questions and cover extensive information in the system card. This involves a deeper understanding of the model’s behavior and why it is exhibiting those behaviors.
Benchmarks Every Technical Writer Should Know
Technical writers need to be familiar with methodologies of popular LLM benchmarking tests for evaluating various LLM capabilities. This includes
- SWE – Bench verified: A human-validated subset of real GitHub issues where the model must produce a code patch that passes the repository’s actual tests, measuring practical software-engineering ability
- MMLU: A multiple-choice test spanning 57 subjects from history to law to maths, measuring broad general knowledge and reasoning
- Humanity’s Last Exam: a deliberately extreme benchmark of thousands of expert-written questions across many disciplines, designed to stay hard for frontier models
- SuperGPQA: A large graduate-level multiple-choice benchmark covering hundreds of specialized disciplines.
- AA Omniscience: Artificial Analysis’s benchmark that measures factual knowledge across many topics while also penalising confident wrong answers, so it captures both what a model knows and whether it hallucinates
It also helps to refer to other providers’ system cards and cover sections not addressed in their own.
Conclusion
A system card of the frontier model is a documentation artifact. The regulatory mandate across the world is empowering frontier labs to publish system cards to bring transparency into what LLMs can do and their limitations. This system card helps system integrators and forward deployment engineers to design AI-based solutions and assess the right LLM for their use cases. System card documentation is also evolving as LLMs improve at reasoning and agentic capabilities. Thus, technical writers need to constantly conduct research and keep up with emerging technologies to document them more effectively.
