API technical and data standards

These​​These standards are for people building APIsApplication Programming Interfaces (APIs) in government who want to:

• save time

• save and resources

• reassure

  assure users that their service meets minimum standards

• use agile methods to improve products and services

• use throughthe testingREST API style

• take an API-first approach to development

TheseYou standardsshould willuse providethese youstandards withwhen bestdesigning, practicebuilding guidance about how to design, build and operateoperating your APIs for use in government and public services.

By followingThis thesewill standards,ensure you’llthat workyour in the same way as other people and produce APIs that work better across different platforms and services.

Design your API

Gather user needs

Before you build an API, you shouldmust carefullyunderstand planthe whatneeds itof willyour dousers.

For andan howAPI, itthe willuser work.is Your team should agree on a setdeveloper ofwho userwants needs,to includingconsume whoyour willAPI useto yourdeliver API,a andservice. whatThe theydeveloper will needhave toneeds dobased with it.on:

Start by:

• gathering user and business requirements to help define what

  the APIservice needsthey toare dodeveloping

• identifying

  how theeasy keyit businessis entities users need to interactwrite withcode via the API

• developing a specification before you start to code
• testingconsume your assumptions with users
API
• iterating the design based on feedback

ThisMaking willyour allowAPI youeasy to focusunderstand onmeans simplifyingit’s the interface, removing any features that are not useful to users.

Consistency improves the developer experience by making it easier and more intuitivelikely to integratebe withused, thebecause API.developers You need to make your API as easy to predict as possible as users may not read all of your documentation.

YouStarting should avoid introducing breaking changes. Once an API is integrated with auser service,needs people may be less willing or unable to update their code, because they will oftenallow wantyou to makefocus as few changes as possible after it goes live. You should think about services which depend on yoursimplifying APIthe andinterface, howremoving changesany willfeatures impactthat them.are If possible, do not makeuseful changes that will change how services have to interactusers.

Check withfor theexisting API.

APIs

ImplementingIt’s user-centredfaster designand makessimpler itto morereuse likely an existing API willthan meetbuild userone needsfrom andscratch. needYou fewershould changesonly overbuild itsa lifetime.new If you do need to introduce breaking changes, follow our guidance on versioning and taking your API outwhen of service.necessary. 

Check

You forcan existingcheck APIs

Establishif whether there are any existing APIsinternal, whichexternal could be used instead of building a new one. These might be internal or external, or commercially available.available YouAPIs shouldby looklooking at internal API catalogues and the cross-government UK API Catalogue.

It’sMake fastersure andthat simplerany to reuse an existing API thanyou buildchoose onewill fromhave scratch. Building this discovery step into the designright processfunctionality prioritises reuse and means APIs are built only when necessary. You should also make sure any API you choose to use will work for your use case,case. inYou termsmust ofalso licensingcheck and functionality.

Follow the Technologylicence Codeit ofis Practice,provided governmentunder, data standards and all legal requirements

You should follow the Technology Code of Practice when designing any technology in government, including APIs. Point 10 - Make better use of data - may be particularly relevant when you’re working with APIs.

You should design your API to followmake allsure appropriateyou government data standards. You can find a list of endorsed standards in the Data Standards Catalogue. Certain data standards endorsed by the government are particularlyable usefulto inuse APIit design. They are the:

• ISO 8601 standard, which represents the date and time in your API’sservice. payloadFor response,example, helpingan peopleopen toAPI readprovided theunder timean correctly
• Worldopen Geodetic System 1984 and European Terrestrial Reference System 1989 standardslicence, whichcan helpbe youused exchangewith locationfew informationrestrictions, inbut government
• GeoJSONcommercial formatAPIs formay encodinghave andusage exchanging location information

limits.

YouRead shouldmore followabout all guidance and regulations around the General Data Protection Regulation (GDPR) and using personalopen datalicences infor yourgovernment organisationservices.

PracticeDesign API-firstyour designAPI first

API first is the practice of designing software starting with an APIAPI, asbefore thedesigning first interface to your data - and then having further interfaces such as web or mobile useuser that API as a way to access the data.interface.

Developing the API before the rest of the service means a platform or service can be built around the API. This wouldwill reduce the need to repeat workwork, if later ifon an external API is required.required for your service.

APIs are often an afterthought,afterthought – built when a service already existsexists, as a way for other services to access its data. If you’re building an API to a legacy system, that may be your only choice, but you should still think about the user needs for your API.

Developing your API before any other interfaces has other advantages.advantages, These include:including: 

• other services being able to use your API

• stress-testing of your API by your own internal services, allowing you to make continuous improvements to– it and improve it for external users - this will improve developer experience by exposing the complexities of the API and making sure, for example, that documentation is fit for purpose

• improved modularity and reuse of code, as the API will not have to be customised to fit an existing service -– this leads to more consistent interfaces, meaning developers can be more comfortable with your API and can speed up integration

• the underlyingresources data structures of your API exposes will be fit for purpose -– by starting with the APIAPI, it means that the designbusiness logic of your service can be clearly separated from the data structures used in any underlying data store

Follow mustthe beTechnology consideredCode fromof thePractice beginning, and separatedother fromstandards

You should follow the businessTechnology logicCode of Practice when designing any technology in government, including APIs. Point 10 – Make better use of data may be particularly relevant when you’re working with APIs.

You also need to design your API to follow all appropriate government data standards in the serviceData (seeStandards belowCatalogue.

The ‘Abstractfollowing awaystandards are especially useful:

• ISO 8601 standard – this represents the date and time in your processes’)API’s responses, preventing issues with ambiguous date formats

• GeoJSON – use this format for encoding and exchanging location information

Use

The RESTUK toGeospatial buildData Standards Register has details of coordinate reference systems that you should use when exchanging location information through your API

API.

RepresentationalYou stateshould transferalso (REST,follow whichthe isguidance sometimesrelated knownto:

• UK asGeneral RESTful)Data Protection Regulation (GDPR)

• privacy and data protection by design

Use the REST API style

Representational state transfer (REST) means an API follows the REST architectural style,style, and works with RESTfulREST (sometimes called RESTful) web services. This can make the API easier to use and faster to integrate with.

REST principles are industrywidely standard,adopted, which means developers will understand themyour API more quickly. It also means you can use industryindustry-standard standard ways of documenting your API, and offoff-the-shelf testing software.

When you’re using the shelfREST testingAPI software.style, you should follow the REST design principles:

• uniform interface – all API requests for the same resource should have the same uniform resource identifier (URI)

• client and server – these must be independent of each other

• statelessness – this means that all requests and responses are self-contained and include all necessary information, and that no server-side sessions are required because all session state is kept on the client side

• cacheable – resources should be cached on the client side and server side, to improve performance and scalability

• layered system – allows for components such as proxies, gateways and firewalls to be placed between the client and the server, to make the service more reliable and secure

Using REST is a good way to builddesign an API, but other approaches such as GraphQL or gRPC may still be a better choice for specific projects. For example, GraphQL is useful for prototyping services, when you’re not sure what views of the data other developers will need.

You should always choose the architectural style for your API based on your individual project or service’s needs.

ProduceDevelop ana OpenAPIspecification documentbefore foryou yourstart APIto code

The OpenAPI Specification is a standardised way of describing RESTful web APIsAPIs, -and itis recommended by the government Open Standards Board.

It allows you to produce a file (the OpenAPI document) which is both machine and human readablereadable, and which describes the format and responses of your API.

A good principle is to produce an OpenAPI document as the first output of your design process, soand then develop the document is developed along with the API design.

This canwill behelp used to:

• show the API has been developed consistently with others from your organisation

• test itthe API against rules and for security issues using a software linter

• generate reference documentation automatically -– documentation generated in this way should be supported by further resources.

OpenAPIFor isnon-REST recommendedAPIs bylike GraphQL, you should still look to produce a specification during the governmentdesign Openprocess. StandardsFor Board.example, for GraphQL this would be a GraphQL schema specification defining the types available, queries, mutations and the relationships between them.

PlanBe yoursecure APIby securelydesign

Design securely

You shouldmust think about security from the very beginning of the design process for your API.API, Theand Nationalfollow Cybera Securitysecure Centreby providesdesign advice about designing services securelyapproach.

API security is complicated, and involves:

• data level security -– making sure users only have access to the data provided by the API that they are authorised to see

• application level security -– making sure only authorised users can access the API

• network security – making sure only trusted clients can consume your API

• auditing -– making sure the usage of the API is monitored

The OWASP foundation has a list of top 10 security risks for APIs. YouMake should make sure toyou avoid these when designing your API.

Host

Test your APIassumptions securely

with users

WhenBy creating your OpenAPI specification first, before building your API, you can use tools that automatically generate test stubs of your API.

Test stubs allow you to:

• test your design anchoices API,with it’sthe importantpotential toconsumers thinkof aboutyour whereAPI

• build out a suite of functional tests for your API

This can shorten the feedback loop and allow you willto hostrefine ityour API in the design stage without spending time and howeffort itbuilding out the real API. It will continuealso help you follow a test-driven development (TDD) approach to workbuilding duringyour itsreal lifecycle.API.

WhenThere youare namemany different tools that can help when designing and hosttesting your API,API. asSome wellpopular astools itsare:

• SwaggerHub

• Swagger.io

• Postman

• Bruno

• ReadyAPI

All documentation,have youa shouldfree followtier, guidancebut oncharge choosingfor anmore APIenterprise domainfeatures.

There nameare also many open source tools that are free to use – for example, the openapi.tools GitHub repository. maintains lists of open source API tooling. 

FollowWhen using any cloud-based API tool, be sure to check how the Servicetool Manualwill guidancekeep your API credentials secure before sharing them.

Iterate the design based on usingfeedback

Follow HTTPSan agile process when servingdesigning your APIAPI. overYou do this by incrementally building out your design and continuously testing it and getting feedback from the web,people who will be consuming it.

At the design stage it’s easy to make itchanges asto secureyour asAPI. possible.You Thecan Nationalentirely Cyberremove Securityendpoints Centreand alsoredefine hastheir guidanceresponse onformats without worrying about breaking changes.

Once your API is built and people start using TLSit, it becomes much harder to securelymake deliverbig webchanges services.because you need to consider backward compatibility between versions of your API.

Build your API

Use the UTF-8 standard to encode your API

Unicode is the world standard for consistently encoding, representing and handling text in most global writing systems.

You should use the Unicode Transformation Format (UTF-8) standard when encoding unicode character sets,sets. toThis will help you read, write, store and exchange text that will remain stable over time and across different technologies.

Use JSON for response formats

UseWhere possible, you should use the JSON standardData whereInterchange possibleStandard  when structuring your REST API’s response formats.

You should use the official specification for JSON - ECMA-404 The JSON Data Exchange Standard. There are multipleseveral JSON formats in use today,today. andIf if your organisation has not already specified one, we recommend JSON:API. whichThis is specifically designed for API responses, and has the benefits of making some design choices for you by specifying conventions.

SometimesIf you’llyou needare anworking alternativewith toa RESTlegacy toAPI matchthat adoes datanot structureprovide toJSON aresponses specific– usefor case. For example, ifa you’reSOAP updating an API whichthat currentlyuses provides responses in XML, it may be better for your users if you keep the same format. You should make sure this is well documented,documented because XML is increasingly uncommon.

Choosing a broadly-adopted standard whenever possible gives you advantages,the suchadvantages as:of:

• saving time by avoiding debate about what format to use

• allowing you to follow industry best practices

• making it easier for external developers to integrate with your API

Use consistent names for resources

Developers should be able to assume the names of the resources in your API from context.context, Useso similarname termseach type of resource consistently. For example, if a resource represents a collection, choose whether this should always be singular or plural (for example: order or orders).

You should also use naming conventions for similar resources,resources. For example, if you have a user and address resource, the name you use for examplethe id of each should match: user_id and address_id.

Your users should not have to reread documentation to be able to know what the name of a particular resource is.

Make your resources persistent

The names of the resources your API provides (answers to requests) should not change,change between versions, as this could break integrations.

YouMake should make sure your API design provides a level of abstraction from the underlying data sources. It should not matter if the columns in your database change name, for example, because the API should provide a map from the resource name to the underlying data.

Use standard HTTP responses

YouMake should make sure you match error codes with HTTPstandard responses. Standard HTTP response codes are defined here.

ConsistentYour error codes must be consistent and easy to readread, errorso codesthat areit’s crucialclear to understanding where an error has occurred. There are often cases where the same API endpoint could return the same HTTP status code for different conditions, so descriptive error messages will help users understand what’s gone wrong.

You should document all error codes and make sure they’re easy to find.

MakeAny sure any custom error codes should only contain information needed to diagnose the problem,problem. andDo do not include any non-essential information,information which could help an attacker target the service.service For– for example, technical details of the system the API is running on.

Host your API

When you build an API, it’s important to think about where you will host it and how it will continue to work during its lifecycle.

When you name and host your API, as well as its documentation, you should follow guidance on choosing an API domain name.

Control levelsaccess ofto useryour authorisationAPI

When you’reyou buildingbuild your API, you need to decide the best way to authorisegive peopleusers toaccess. accessIn itsgeneral you should make all users of your API authenticate their identities. This is essential if your API deals with personal or sensitive data.

Avoid Readendpoints that allow anonymous users, as these can:

• increase the APIattack managementsurface guidanceavailable to hackers.

• make it harder to monitor users that are consuming excessive resources 

User-levelIf authenticationyou isrequire goodanonymous forendpoints, audittheir responses should be limited to open data. For example the GOV.UK Content API does not require authentication, but only returns metadata and accessthe control.HTML content of pages published on GOV.UK. Anonymous endpoints should also be rate limited to prevent excessive or malicious use.

Use the industry standard OAuth 2.0 Authorization Framework to manage access to your API. This will make it ifeasier youfor wantusers to consume your API while giving you better control whoover canthe level of access yourthey API.have. 

Never use basic authentication because usernames and passwords are sent encoded, but unencrypted, in the HTTP header. This makes it easy for an attacker to steal them. 

You should also avoid using API keys. An API key is essentiala whenunique dealingidentifier that is issued to API users. It needs to be sent with personalevery request – either in the URL, as a request header or sensitiveas data.a cookie – and it can easily be intercepted by an attacker and reused. If you do use API keys you should time limit their use and regularly change the keys to keep them secure.

UseNeither application-levelbasic authorisationnor ifAPI key authentication is secure unless used in conjunction with HTTPS. 

OAuth 2.0 typically uses digitally signed security tokens in JWT (JSON Web Token) format that are passed in the Authenticate header of a request, making them harder to tamper with.

OAuth 2.0 defines ways to authenticate different types of API clients:

• use client credentials when another service or application is consuming your API, outside of the context of a user

• use authorization code, with the PKCE extension, when a user accesses your API – for example, through a web application

As well as authenticating your users, you wantshould define resource-level access controls for your API, and check the authorisation for every request. 

When using OAuth 2.0 this means:

• a user must first request the scope of access they need to your API – for example, an order-read or order-write scope

• on each call to your API you should check that the user has the required scope before granting them access

Defining scopes in this way allows for fine-grained access control whichto applicationsthe endpoints of your API, and means you can accessinclude the authorisation information in your OAS specification. This makes it easier for consumers to use your API withoutsecurely.

Authentication limitingand whoauthorisation are features that can accessbe them.defined and controlled through an API management system. Find out more about how to manage operations with an API gateway.

Secure your API

Follow the GOV.UK Service Manual guidance on using HTTPS when serving your API over the web, to make it as secure as possible.

You must use TLS 1.2 or above to secure your API. The National Cyber Security Centre has guidance on using TLS to securely deliver web services.

Validate all inputs to your API. Endpoints that de-serialise data should enforce schema validation and reject unknown attributes. All parameters (URL and query string) should be validated and type checked before being processed. Otherwise attackers could exploit lax input validation to craft requests that result in unexpected effects.

Configure your API using appropriate Cross-Origin Resource Sharing (CORS) headers. This iswill suitableminimise the risk of cross-origin attacks.

Turn off unnecessary HTTP verbs (GET, PUT, POST, etc), and only allow the verbs your API actually supports on each resource. For example, if youusers wantcan tonever be deleted, do ratenot limiting,accept auditing,DELETE oron billingthe functionality.users Application-levelresource.

Remove authorisationany mayendpoints you do not beneed suitable– for APIsexample, holdingtest personalendpoints or sensitivestubs data.for future development – and only expose the endpoints users will actually use.

You can also enforce your content type. For example, if your API only accepts JSON then enforce this through content type headers and reject other types of request.

For a complete list of common security issues to fix in your API, refer to the OWASP API Security Project.

Consider performance and scalability

You can measure your API’s performance by how fast it can deal with a single request and make a response. Its scalability is the amount of requests it can deal with at the same time while maintaining an acceptable performance level.

You can improve the performance and scalability of your API response by making it cacheable. This means the API response can store copies of frequently-accessed data along its request and response path. This reduces bandwidth, latency and server load, as well as making network failures less of a problem for your users.

A cacheable API response has other advantages, for example letting you use a Content Delivery Network (CDN). A CDN can make your API faster and more reliable by caching data for responses in locations that are closer to the user.

You should also implement rate limiting or throttling policies – making sure that users don’t overuse your resources

Provide an API test service

You should try to offer a testing service (also known as a sandbox) for your users. They may find an API harder to integrate without a test service, so providing one is a good use of your project’s time and budget.

Sandboxes can also be generated much more easily with an OpenAPI documentdocument. -There tools are tools available whichthat will convert these documents into test endpoints.

YouAPI test services should makestill yourbe testsecure servicesand availablerequire withoutall authenticationusers andto makebe sureauthenticated thebefore granting access. Many API management solutions include developer portals that allow developers to create development accounts which they can use to access test serviceservices.  

Test services should never containscontain real data.personal or transactional data, but could use real reference data – for example, lists of local authorities.

Having a test service available also means your developers can can:

• start to get comfortable with an API in a sandbox environment very early in its development. It means your developers can development

• work while other parts of the project are being completed.completed For– for example, workthey can beginwork inwith a test service,data within the test data,service while a data sharing agreement is still being drawn up to access the real data for the finished API.API

However, providing a sandbox of good quality can be complicated,complicated soso  you should carefully consider your options carefully if youyou’re are going to do it. For example, if you plan to create synthetic data, you may need to use a paid service.

A good sandbox should include:

• useful test data that reflects the real API

• implementation and dependencies behaviour that is properly simulated

• the ability to save, protect and restore data

Test your API’s compliance

You needmust to make sure your API meets the standards your organisation or team’s legal requirements need.

Youof mustyour comply with UK GDPR when you do this.organisation. 

Put your API in the cross-government API Catalogue

After you’ve tested your API’s compliance, you should addmake yourit APIeasy to thefind cross-governmentby APIadding Catalogue.it Makingto your organisation’s API discoverablecatalogue meansand it’sthe morecross-government likelyAPI toCatalogue, getso used.that others can use it. 

Document your APIs

Your team’s developer, user researcher and technical writer should work together on your API’s documentation.

Follow Youthe should follow guidance on:

• documenting APIs

• writing API reference documentation

At the end of the build phase, it’s always useful to carry out a quick evaluation. Some of the questions you might ask include:

• was UTF-8 used for all text encoding?

• how were dates and times represented?

• how is user-level authorisation managed?

• what are the limits of scalability of your API?

• is the documentation easy to understand?

Operate your API

SupportVersion olderyour versionsAPI ofand yoursupport APIolder versions

When you make new versions of your API, you should try not to make sure you do not make changes that will stop older versions of your API working properly. If you cannot keep older versions working,working you should tellexpose usersa thesenew olderversion versionsof areyour noAPI longerby supported.

Youadding canthe thinkversion aboutnumber retiringinto olderthe versionsURI, offor anexample APIhttps://myapi.service.gov.uk/v1. 

URI ifversioning youis canthe tellsimplest fromand themost logscommonly thatused fewway peopleto areversion usingan it anymore.API. 

Use

Other ways to version an API managementinclude systemusing ora gateway

Ancustom APIheader managementor platformdefining providesa servicescustom formedia yourtype. APIAvoid thatthese itapproaches rarelybecause makesthey sensecan forlead you to buildyour yourself.API Forbeing example,blocked accessby controlproxies andor authorisation,firewalls.

You auditshould andtry logging,to andkeep networkthe management.number Inof productionactive allversions of theseyour servicesAPI areto important,a minimum, and canencourage beusers moreto reliablymove andto easilythe providedlatest byversion, anto APIreduce managementthe tooloverhead or gateway - of whichmaintaining theremultiple are several open source options.versions.

When to authenticate your API

AuthenticationYou allowscan youthink toabout monitorretiring whoolder isversions usingof youran API,API andif foryou whatcan purpose.tell Youfrom needthe tologs authenticatethat youronly APIa tofew use:

• ratepeople limitingare orusing throttlingthem -anymore. makingWhen sureyou thatretire usersan don’told overuseversion youryou resources
• auditingmust -tell checkingusers whatit datais wasno accessedlonger bysupported whichand users
• billinggive -them iftime yourto APImove ison chargeable
• authorisationto -the enablingnew differentversion.

  Use levelsan ofAPI accessmanagement basedsystem on users or roles

gateway

WeAn recommendAPI usingmanagement aplatform thirdprovides partyservices tool for authentication,your either as part of or in conjunction with an API gateway.that Severalit openrarely sourcemakes optionssense arefor available.

It’syou unlikely you’ll ever need to providebuild completelyyourself. openFor example, access tocontrol your API, and forauthorisation, securityaudit reasonsand it’slogging, neverand anetwork goodmanagement.

In ideaproduction toall doof so.these Ifservices youare doimportant, wantand tocan providebe openmore datareliably inand thiseasily way,provided youby shouldan considerAPI publishingmanagement ittool asor agateway CSV– fileof towhich thethere internet,are forseveral exampleopen onsource data.gov.uk.options.

Log your API’s use

If your API provides personal or sensitive data, you must log (ideally using the functionality of an API gateway, as above) when the data is provided and who you provide it to.

This will help you you:

• follow UK GDPR,

• respond to data subject access requests, and requests

• detect fraud or misuse.misuse

An API management tool can help with logging and auditing usage of your API.

Monitor APIs for unusual activity

You must make sure all technology in your team or organisation is secure. APIs are no exception. ReadThe guidance from the Technology Code of Practice toexplains helphow youto make things secure.secure. Using the logging and audit tools from your API gateway will help you recognise when use of the API changes over time.