<\/a><\/p>\nThey also emphasise context<\/span><\/p>\n“Context is probably one of the most important aspects of ethical and responsible machine learning. This is because, despite it being talked about as an independent phenomena, machine learning is – arguably – an augmenting technology. It augments the process and\/or operations it is applied in. This means it is a tool (means), as opposed to an end-product (ends). Given this, the context of any machine learning operation is very important in understanding how best and responsibly this technology can be used\u00a0<\/em>and what its particular risks might be<\/em><\/span>.”<\/em><\/p>\n\n
\n<\/div>\n
Themes covered<\/strong><\/div>\n\u00a0Managerial Oversight & Management<\/div>\n
\u00a0Internal Organisation\u00a0<\/div>\n
Management & Oversight<\/div>\n
Data Governance<\/div>\n
Product and Model Oversight & Management<\/div>\n
Product Validation<\/div>\n
Human Resources Management<\/div>\n
Asset Management<\/div>\n
Software Management<\/div>\n
Incident Management<\/div>\n
Third Party Contracts Management<\/div>\n
Ethics & Transparency Management<\/div>\n
Compliance, Auditing & Legal Management and Oversight<\/div>\n
<\/div>\n
Definitions and terminology for ethical AI and responsible AI (from the best practice wiki)<\/strong><\/div>\n\n- Absolute Reproducibility\u00a0means a guarantee that any and all results, outputs, outcomes, artifacts, etc can be exactly reproduced under any circumstances.<\/li>\n
- Adversarial Action\u00a0means actions characterised by mala fide (malicious) intent and\/or bad faith.<\/li>\n
- Assessment\u00a0means the action or process of making a series of determinations and judgments after taking deliberate steps to test, measure and collectively deliberate the objects of concern and their outcomes.<\/li>\n
- Assets\u00a0means information technology hardware that concerns Products Machine Learning.<\/li>\n
- Best Practice Guideline\u00a0means this document.<\/li>\n
- Business Stakeholders\u00a0means the departments and\/or teams within the Organisation who do not conduct data science and\/or technical Machine Learning, but have a material interest in Products Machine Learning.<\/li>\n
- Confidence Value\u00a0means a measure of a Model’s self-reported certainty that the given Output is correct.<\/li>\n
- Corporate Governance Principles\u00a0mean the structure of rules, practices and processes used to direct and manage a company in terms of industry recognised and published legal guidelines.<\/li>\n
- Data Generating Process\u00a0means the process, through physical and digital means, by which Records of data are created (usually representing events, objects or persons).<\/li>\n
- Data Governance\u00a0means the systems of governance and\/or management over data assets and\/or processes within an Organisation.<\/li>\n
- Data Quality\u00a0means the calibre of qualitative or quantitative data.<\/li>\n
- Data Science\u00a0means an interdisciplinary field that uses scientific methods, processes, algorithms and computational systems to extract knowledge and insights from structured and\/or unstructured data.<\/li>\n
- Domain\u00a0means the societal and\/or commercial environment within which the Product will be and\/or is operationalised.<\/li>\n
- Edge Case\u00a0means an outlier in the space of both input Features and Model Outputs.<\/li>\n
- Error Rate\u00a0means the frequency of occurrence of errors in the (Sub)population relative to the size of the (Sub)population<\/li>\n
- Ethical Practices\u00a0means the ethical principles, values and\/or practices that are encapsulated and promoted in an ‘artificial intelligence’ ethics guideline and\/or framework, such as (a) The Asilomar AI Principles (Asilomar AI Principles, 2017), (b) The Montreal Declaration for Responsible AI (Montreal Declaration, 2017), (c) The Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems (IEEE, 2017), and\/or (d) any other analogous guideline and\/or framework.<\/li>\n
- Ethics Committee\u00a0means the committee within the Organisation charged with managing and\/or directing organisation Ethical Practices.<\/li>\n
- Evaluation Error\u00a0means the difference between the ground truth and a Model’s prediction or output.<\/li>\n
- Executive Management\u00a0means the managerial team at the highest level of management within the Organisation.<\/li>\n
- Explainability\u00a0means the property of Models and Model outcomes to be interpreted and\/or explained by humans in a comprehensible manner.<\/li>\n
- Fairness & Non-Discrimination\u00a0means the property of Models and Model outcomes to be free from bias against protected classes.<\/li>\n
- Features\u00a0mean the different attributes of datapoints as recorded in the data.<\/li>\n
- Guide\u00a0means an established and clearly documented series of actions or process(es) conducted in a certain order or manner to achieve particular outcomes.<\/li>\n
- Hidden Variable\u00a0means an attribute of a datapoint or an attribute of a system that has a causal relation to other attributes, but is itself not measured or unmeasurable.<\/li>\n
- Human-Centric Design & Redress\u00a0means orienting Products and\/or Models to focus on humans and their environments through promoting human and\/or environment centric values and resources for redress.<\/li>\n
- Implementation\u00a0means every aspect of the Product and Model(s) insertion of and\/or application to Organisation systems, infrastructure, processes and culture and Domains and Society.<\/li>\n
- Incident\u00a0means the occurrence of a technical event that affects the integrity of a Product and\/or Model<\/li>\n
- Label\u00a0means the Feature that represents the (supposed) ground-truth values corresponding to the Target Variable.<\/li>\n
- Machine Learning\u00a0means the use and development of computer systems and Models that are able to learn and adapt with minimal explicit human instructions by using algorithms and statistical modelling to analyse, draw inferences, and derive outputs from data.<\/li>\n
- Model\u00a0means Machine Learning algorithms and data processing designed, developed, trained and implemented to achieve set outputs, inclusive of datasets used for said purposes unless otherwise stated.<\/li>\n
- Organisation\u00a0means the concerned juristic entity designing, developing and\/or implementing Machine Learning.<\/li>\n
- Outcome\u00a0means the resultant effect of applying Models and\/or Products.<\/li>\n
- Output\u00a0means that which Models produce, typically (but not exclusively) predictions or decisions.<\/li>\n
- Performance Robustness\u00a0means the propensity of Products and\/or Models to retain their desired performance over diverse and wide operational conditions.<\/li>\n
- Policy\u00a0means a documented course of normative actions or set of principles adopted to achieve a particular outcome.<\/li>\n
- Procedure\u00a0means an established and defined series of actions or process(es) conducted in a certain order or manner to achieve a particular outcome.<\/li>\n
- Product\u00a0means the collective and broad process of design, development, implementation and operationalisation of Models, and associated processes, to execute and achieve Product Definitions, inclusive of, inter alia, the integration of such operations and\/or Models into organisation products, software and\/or systems.<\/li>\n
- Product Lifecycle\u00a0means the collective phases of Products from initiation to termination – such as design, exploration, experimentation, development, implementation, operationalisation, and decommissioning – and their mutual iterations.<\/li>\n
- Product Manager\u00a0means either a Design Owner and\/or Run Owner as identified in the Organisation Best Practice Guideline in Sections 3.1.4. & 3.1.7. respectively.<\/li>\n
- Product Owner\u00a0means the employee charged with (a) managing and maximising the value of the Product and its Product Team; and (b) engaging with various Business Stakeholders concerning the Product and its Product Definitions.<\/li>\n
- Product Subjects\u00a0means the entities and\/or objects that are represented as data points in datasets and\/or Models, and who may be the subject of Product and\/or Model outcomes.<\/li>\n
- Product Team\u00a0means the collective group of Organisation employees directly charged with designing, developing and\/or implementing the Product.<\/li>\n
- Project Lifecycle\u00a0means the collective phases of Products from initiation to termination – such as design, exploration, experimentation, development, implementation, operationalisation, and decommissioning – and their mutual iterations.<\/li>\n
- Protected Classes\u00a0mean (Sub)populations of Product Subjects, typically persons, that are protected by law, regulation, policy or based on Product Definition(s)<\/li>\n
- Public\u00a0means society at large.<\/li>\n
- Public Interest\u00a0means the welfare or well-being of the Public.<\/li>\n
- Representativeness\u00a0means the degree to which datasets and Models reflect the true distribution and conditions of Subjects, Subject populations, and\/or Domains.<\/li>\n
- Root Cause Analysis\u00a0means the activity and\/or report of the investigation into the primary causal reasons for the existence of some behaviour (usually an error or deviation).<\/li>\n
- Safety\u00a0means real Product Domain based physical harms that result through Products and\/or Models applications.<\/li>\n
- Security\u00a0means the resilience of Products and\/or Models against malicious and\/or negligent activities that result in Organisational loss of control over concerned Products and\/or Models.<\/li>\n
- Selection Function\u00a0means a (where possible mathematical) description of the probability or proportion of all real Subjects that might potentially be recorded in the dataset that are actually recorded in a dataset.<\/li>\n
- Social Corporate Responsibilities\u00a0means the structure of rules, practices and processes used to direct and manage a company in terms of industry recognised and published legal guidelines to positively contribute to economic, environmental and social progress.<\/li>\n
- Software\u00a0means information technology software that concerns Products Machine Learning.<\/li>\n
- Special Interest Groups\u00a0means a specific body politic, or a particular collective of citizens, who can reasonably be determined to have a material interest in the Product.<\/li>\n
- Specification\u00a0means the accuracy, completeness and exactness of Products, Models and\/or datasets in reflecting Product Definitions, Product Domains and\/or Product Subjects, either in their design and development and\/or operationalisation.<\/li>\n
- Stakeholders\u00a0mean the department(s) and\/or team(s) within the Organisation who do not conduct data science and\/or technical Machine Learning, but have a material interest in Product Machine Learning.<\/li>\n
- Subjects\u00a0means the entities and\/or objects that are represented as data points in datasets and\/or Models, and who may be the subject of Product and\/or Model outcomes.<\/li>\n
- (Sub)population\u00a0means any group of persons, animals, or any other entities represented by a piece of data , that is part of a larger (potential) dataset and characterized by any (combination of) attributes. The importance of (Sub)populations is particularly high when some (Sub)populations are vulnerable or protected (Protected Classes).<\/li>\n
- Systemic Stability\u00a0means the stability of Organisation, Domain, society and environments as a collective ecosystem.<\/li>\n
- Target Variable\u00a0means the Variable which a Model is made to predict and\/or output.<\/li>\n
- Target of Interest\u00a0means the fundamental concept that the Product is truly interested in when all is said and done, even if it is something that is not (objectively) measureable.<\/li>\n
- Traceability\u00a0means the ability to trace, recount, and reproduce Product outcomes, reports, intermediate products, and other artifacts, inclusive of Models, datasets and codebases.<\/li>\n
- Transparency\u00a0means the provision of an informed target audiences understanding of Organisation and\/or Products Machine Learning, and their workings, based on documented Organisation information.<\/li>\n
- Variables\u00a0mean the different attributes of subjects or systems which may or may not be measured.<\/li>\n
- Workflows\u00a0means the coordinated and standardised sequences of employee work activities, processes, and tasks.<\/li>\n<\/ol>\n