Introduction
In any API management solution, such as IBM API Connect, developers need to create APIs that must comply to architected application specifications. This means taking consumer requests, format mapping to downstream services, invoking those services and then shaping the responses into consumer-friendly formats and data compliant responses. They must also ensure they are meeting any regulatory or organisational data compliance requirements, such as Open Banking or GDPR.
Organisations must also be attentive to the data compliance rules that must be followed when ingesting and storing data. Data containing personally identifiable information (PII) should not be stored unless it is for legitimate business reasons – for example, personal information being retrieved from a CRM system and logged in plain text in log files. Organisations must further respect data retention periods and the right to be forgotten for any data they store and process.
To protect this data and meet data compliance rules, three common techniques are employed by delivery teams:
Payload scrubbing – removing data that should not be passed to either an invoked backend service or back to the consumer.
Backend specific – only send data that is needed by the backend, typically used with a switch statement in IBM API Connect.
Location specific – remove data that should leave a specific geography or may not be pushed between geographies e.g., national insurance numbers in the UK and US social security numbers.
User authorization – ensuring data returned in response to consumer requests is appropriate to their authorization credentials e.g., only data approved for marketing purposes is supplied to a marketing team.
Redaction – removing or obfuscating sensitive data to either stop data being logged or showing only a data representation e.g., instead of a postcode, showing XXXX 9PJ.
Encryption/Pseudonymisation – persist and manage data in a non-human readable format which can be reconstructed later as per the entitlements associated with the caller.
To achieve these use cases today using IBM API Connect, API Developers either need to write and embed custom gateway-scripts, create custom mappings, use XSLT or redaction policies in each API Connect proxy.
Compliance Challenges when embedding Compliance and Data Governance into APIs
Including data governance policies and data privacy principals inside the API code raises several considerations that increase development, testing and maintenance effort, which results in increased complexity and cost:
Requires an enterprise data architecture to review the application design and raise requirements e.g., all payload data must be redacted.
Enterprise data policies need to be embedded and repeated in each proxy for each data distribution use case.
Developers need to understand and implement the best method for performing payload scrubbing.
Developers need to understand how to create JSONata expressions in redact policies and apply those consistently.
Data governance needs to be performed to ensure no data formats, attributes or objects have been missed.
When data rules change or evolve e.g., what data can be shared out of country, then all APIs need to be updated, tested, and released to support the use cases.
APIs may need to be updated in the event of API Connect upgrades.
Simplify Data Governance with eXate APIgator & Co-pilot
eXate is a data governance and privacy company that provide services to secure enterprise data on the fly and at rest. Essentially, you configure re-usable data privacy rule-based policies at an enterprise level and all API traffic is checked against those policies.
Fundamentally eXate takes a dataset, applies centralised data entitlements, and returns a sanitised payload for further processing based upon the context of the request.
Some simple examples of the types of entitlements that eXate can be applied to include:
Which user groups can have access to which attributes and object in a dataset.
Which data to permit to leave or enter specific locations or countries.
Applying the same rules consistently and automatically by detecting data objects that are the same e.g., Pay, Salary, NetPay should be managed consistently.
An immediate benefit is that the data governance policies are now external to the API code, an Organisation’s Data Governance experts manage the governance rules without the need to be constrained by the IT lifecycle. eXate removes the need for large scale IT changes and performing entire regression tests because the governance policies need to be updated.
By plugging eXate into API proxies we do not need to perform payload scrubbing and redaction in each API but can benefit from the enterprise-wide defined rules.
Some of these benefits in API Connect are:
Reduction in development time – there is no longer a need to write and test gateway-script, mapping, XSLT or redaction policies. You just need to call the exate service to apply pre-defined entitlements.
Reduction in project time, complexity and data privacy and security risks
Increased and improved oversight and governance at the enterprise level – instead of checking each APIs implementation, we simply enforce that eXate must be used and automatically have assurance we are meeting the enterprise rules.
This can even be achieved by performing checks in the DevOps pipelines to check for an eXate call in the API YAML.
Lower risk and costs to creating and delivering APIs – we do not need to redeploy the API if data policies change, this saving is multiplied by every API affected by a data change.
Create consistency and drive re-use by re-purposing API contracts to support many use cases.
Co-pilot allows businesses to use eXate's AI tools to automatically build and classify datasets into the eXate meta model at either design or execution time to create automated policy management. Co-pilot can:
Advise designers about the risks associated with the use of data attributes, data types and data objects in an API contract.
Identify data to automatically create and apply data entitlement rules based upon organisational and regulatory policies.
Identify similar data naming, for example, the pay attribute being the same as the salary attribute.
Feed real time data insights downstream to enhance threat detection processes.
Expert Labs eXate Policy
IBM Expert Labs have worked with eXate to create an API Connect custom policy and code free integration pattern to quickly integrate with eXate and show case how eXate can help and benefit API Connect customers. We implemented some commonly seen use cases to quickly highlight the power of centralised data policy enforcement.
Only bank accounts domiciled in the UK because the call is made from the UK (location specific responses)
No address is returned because they are not part of the marketing group (Authorisation specific responses)
Access to electoral roll data from the UK as the request source originates in the UK.
Quickly embed and use eXate policies.
Considerations
eXate is available as a SaaS or on-premises container deployment but is a pre-requisite for centralised policy management and enforcement.
Data governance teams need to create the relevant policies in eXate to realise the benefits of data re-use, reduced risk, and delivery times.
Co-pilot can accelerate the adoption of data rules by learning API traffic from the design stage and pre-production environments.
Implementing eXate may add a little latency to the API traffic in return for lowering data leakage or mis-use risks.
Contact Us
For more information on eXate and API Connect, contact us here.