Co:Driver

The star of our recent releases is Co:Driver, using generative AI to drive a leap forward in how customers interact with Peak’s AI products. We wanted Co:Driver to cover three distinct use cases on release: model output explainability (e.g., why is my recommended safety stock X?), domain understanding (e.g., what is the difference between cycle service level and fill rate?) and analytics support (e.g., tell me where product X is overstocked and where it is understocked?)

In order to maximize the performance of AI systems, it’s critical to use the right tools for the right job. Sometimes this means building an AI system that can leverage the strengths of an ensemble of approaches, to deliver the right results for the right problems. For narrow, well-defined spaces like model explainability, tight control over the presentation of critical information is crucial for adoption and trust. Knowing this, our new release uses deterministic, templated outputs as a foundation. This should satisfy 80% of customer needs, and could also surface a need to look deeper into the data.

For broader inquiries relating to domain understanding, we are using Peak’s substantial knowledge base, developed over years of delivering products and custom solutions in these domains, to augment foundational LLMs. Under the hood, we have designed the application to support a range of foundation LLMs, including the suite available on Amazon Bedrock and OpenAI, through straightforward configuration.

SOTA in this space is fast moving, both in performance and cost, and Peak believes it wise to build with an expectation of needing to change foundation models from time to time. Our initial release achieves suitable results through a relatively straight forward RAG implementation, enabling Co:Driver to ground its responses in this Peak-specific knowledge, but we have also been looking at fine-tuning to see if we can push model performance even further. 

For complex analytical tasks, we’ve implemented an agentic workflow, mostly with LangChain. Agentic workflows define and execute steps in sequence, allowing for multi-step reasoning and data analysis. Our system breaks user queries down into actionable steps, and leverages tools like a restricted SQL engine to navigate our data model and provide answers-on-demand.

The development of viable agentic workflows for solving arbitrary tasks represents one of the most intriguing outcomes of the generative AI boom, and these are likely to completely transform how problems are solved in some industries.

The breadth of scope for Co:Driver is large, and it was no surprise that our tests using a single foundation model to solve all of these problems directly were unsuccessful. In order to manage performance over such a vast space of potential queries, we’ve developed a classification model that routes questions to appropriate response models (or returns templated responses when necessary).

This approach helps us balance the need for accuracy with the flexibility to handle a wide range of user inputs. We’ve also integrated Langfuse for performance monitoring and iteration, making insight into how our AI system is performing in real-world scenarios easier to obtain.

Pricing AI

Our Pricing AI product got off to a fast start in 2024, with significant releases for our Quote Pricing, List Price Optimizer, Markdown and Promotions modules, and re-architecting significant parts of the modules to make the most of powerful new Peak platform features relating to how we manage APIs and product front ends.

In the manufacturing and B2B space, we have incorporated gradient boosted tree models into parts of our Quote Pricing module as an optional configuration. Gradient boosted trees, such as xgBoost and LightGBM, can be particularly attractive for B2B price optimization, for a number of reasons:

• Manufacturing and B2B price data sets are often significantly smaller than retail datasets, and tree-based machine learning models perform exceptionally well in ‘small’-data regimes.
• Manufacturing and B2B price data sets can vary significantly in the types and attributes of data they are capturing, which means optimal model features can vary significantly for different businesses. However, tree-based algorithms are flexible and can accommodate such differences out of the box — providing a robust baseline from which we can tune our models during deployment to improve from.
• Tree-based models are especially effective at capturing the complex, non-linear relationships in price optimization without the need to explicitly account for confounding variables.
• Modern gradient-boosting algorithms make it easy to incorporate monotonic constraints. Monotonic constraints are restrictions applied to models to enforce a specific relationship between the input features and the predicted outcome. This is incredibly useful for price modeling, because the nature of these relationships are usually known in advance (and usually follows the Law of Demand).
• From a system’s performance point of view, this choice in models can speed up the computation step given their ability to run in parallel (during tree construction) so the available hardware can be used more efficiently.

We wrapped up the half with the release of our List Price Optimizer module on Press. This was an amazing effort from the team, progressing from design to release in as little as three months. One of the key challenges in trying to optimize list prices in a B2B business is the ubiquity of set-and-forget pricing strategies in that domain. This results in limited historical price data variance, making it difficult to accurately determine price elasticity or extract meaningful pricing signals.

Normally, our Professional Services team will support configuration of our pricing solutions to ensure we can adequately sample the price-demand surface of our customers’ products, but we wanted the new release of List Price Optimizer to include default sampling approaches out of the box. Unlike a classical multi-armed bandit problem, here we are not trying to optimize our exploration to maximize value; rather we are trying to minimize our uncertainty in the price-demand curve, subject to business guardrail and risk appetite constraints.

Pricing is a very sensitive part of business, and it’s critical for adoption that pricing teams are comfortable with exploration strategies. To this end, and to get started quickly, we have included the ability to sample from defined distributions, including Kumaraswamy distribution and the trapezoidal distribution, due to how easy they are to parameterize to reflect exploration behaviors that are both intuitive and acceptable for our end users.

Outside of manufacturing and B2B pricing, we have ramped an additional squad onto our retail pricing modules, recognizing the unique challenges and opportunities in this space. This team will be instrumental in driving innovation and accelerating our roadmap. One of the first focus areas for this squad was to re-architect parts of Markdown and Promotions’ backend to harness more of Snowflake’s improved Snowpark capabilities, granting our customers the ability to run more of our solution within their own stack if they prefer.

Snowpark has come a long way in a few short years, and it was impressive to see how much of the application backend could be rearchitected to leverage it. Unfortunately (though perhaps not surprisingly), we found the tools available don’t yet support the full range of packages and libraries we require for solving the sorts of metaheuristic, multi-objective optimization problems common in this space, but we have a good partnership with Snowflake and are keeping a keen eye on their new releases.

Another big achievement this year involved bringing our Pricing and Customer products closer together. We have long had the vision of interconnected AI — where businesses can be optimized holistically, and even AI point solutions can perform more effectively thanks to contextual awareness of their place within a broader AI system. This latest release represents a significant stride towards this vision, unifying the underlying retail pricing and customer intelligence data models, and allowing access to core customer intelligence functionality from within our Pricing AI product.

This will allow merchandising teams to collaborate more effectively with marketing teams, leveraging shared insights to drive targeted strategies. For instance, when marketing initiates a reactivation campaign, merchandising can swiftly design and implement promotions tailored to appeal specifically to the characteristics of the identified population of lapsed customers. Conversely, during a merchandising-led close-out event, marketing gains immediate access to data on customers who are in-market for those products. This allows for the deployment of targeted recommendations through optimal channels, enhancing awareness and driving conversion.

Inventory AI

Similar to pricing, we have scaled up our investment in our inventory product, adding an extra engineering squad to help accelerate the roadmap. In addition to this, we have added dedicated deeper-research R&D capacity to help us make sure we are laying the foundations needed to achieve our vision for 2025 and beyond.

We have a few really exciting projects here, supported by collaborations with the University of Manchester and University College London, including a generic framework for simulating arbitrary supply chains that we call the Multi-Echelon Supply SImulation AlgoritHm (Messiah), which we will tell everyone more about soon. Two things it would be nice to share more details about are introducing Fill Rate to our Dynamic Inventory module, and how we have made the module scale much more gracefully.

As mentioned, we made a significant enhancement to our Dynamic Inventory module by introducing fill rate as an optional service level metric available to be configured out of the box, alongside our existing cycle service level. This addition represents a more nuanced approach to measuring inventory policy performance, catering to the diverse needs of our customers.

Cycle service level, our default metric, measures the probability of avoiding stockouts between order replenishments. However, for certain suppliers, particularly those in the consumer packaged goods space, fill rate is a more relevant and useful metric. Fill rate quantifies the proportion of true demand fulfilled between order cycles, providing a different perspective on inventory performance.  The introduction of fill rate optimization addresses a critical issue we observed: when optimizing for cycle service level, the system often recommends holding excess inventory, resulting in a higher-than-targeted fill rate. This discrepancy can lead to inefficiencies and increased carrying costs for businesses that primarily track fill rate.

To implement this feature, we turned to literature, particularly the extensive body of work by Guijarro & Babiloni at the Universitat Politècnica de València. While the fill rate calculations proved to be computationally more intensive than cycle service level, we found this trade-off acceptable given its targeted application in lower SKU-count scenarios. This dual-metric approach allows our customers to choose the service level definition that best aligns with their business model and industry standards, further enhancing the flexibility and precision of our inventory optimization capabilities. It’s another step towards our goal of providing tailored, intelligent solutions that drive real-world business value.

Another fun engineering challenge we have had relates to how our products scale. As Peak has grown, so has the size of our customers, and we now have requirements to be able to generate on the order of a million daily forecasts, and support hundreds of concurrent end business users, all within a single tenant. This growth is a fantastic problem to have, but it has required us to revisit and refine our product architecture.

This was a full team effort, with significant support from both our Professional Services team and our Platform Engineering teams, and the teams rose to the challenge admirably. We are now more careful with identifying and limiting cases of recomputation, and doing this has also helped us identify areas that can be further parallelized. We are leveraging the flexibility of the Peak platform in a better way to allocate additional compute power to specific workflow bottlenecks, and ensure optimal resource utilization.

We utilize a range of time series models, and many of these (particularly statistical ones) can be made incremental, allowing for much more efficient updates as new data becomes available. In the database we have optimized our range joins and are making better use of cluster keys, improving query performance, enhancing retrieval efficiency and reducing data processing times. Finally, it was our Platform team that unlocked the key to improving our front end scale, rearchitecting how we serve APIs and web apps, and handle resource management in general, through Kubernetes.

The cumulative effect of these efforts has been impactful, with Dynamic Inventory data processing and optimization times more than halved. The improvements to the Peak platform obviously impact all our products and customers, and this has resulted in our infrastructure costs relating to these services to be halved as well. This positions us well to handle this higher level of scale and sets a solid foundation for future growth.

Around the team

Beyond our product delivery milestones, the team has been actively engaged in a variety of initiatives that underscore our commitment to improving representation and diversity in tech, and fostering the next generation of AI talent. I firmly believe that to make meaningful progress in this area, we need to raise awareness of the exciting opportunities a tech career offers, help provide clear pathways to success and start these efforts as early as possible in peoples’ careers or education journeys.

One highlight so far was the Peak hackathon, which we successfully hosted for the third consecutive year. This event brought together 48 bright minds from the University of Manchester and Edge Hill University, providing them with hands-on experience in Github skills, data analysis and forecasting. It’s always great to see the creativity and innovation that emerges from these events, and I’d like to extend a special congratulation to Cyrus, Ashley and Samar for their winning contributions, and a special thanks to Kira and Simona for coordinating such a fantastic event. I’d also like to give a shout out to our Data Science mentoring scheme, which is designed to help students build confidence and take control of their own progress in data science and AI.

 

View this post on Instagram

 

A post shared by Peak (@peak_hq)

On the academic front, we’re currently hosting five MSc students (from three universities) who are working on cutting-edge projects, ranging from global optimization methods for inventory policies to active learning for price discovery. This is our eighth year hosting MSc students from the local community, and we’re proud to be able to provide this opportunity for students to develop real-world data science experience. We also collaborated with UCL PhD students from the Centre of Doctoral Training on a group project, using Messiah to explore meta reinforcement learning for supply chain optimization. This went really well, and we’re looking into more exciting ways to collaborate more with UCL moving forwards.

These initiatives, while separate from our core product development, are integral to our mission of making Peak a company that everyone loves being part of.

Conclusion

It has been a busy year so far, and beyond the delivery highlights listed above, improvements to our team structure and our processes have laid the foundations for a really exciting future. If you want more information about any of the above, don’t hesitate to reach out.

• Care about the people in your team and the issues they face. Consider what barriers to inclusion people might have. Look at what is happening in the wider industry
• Listen when people tell you how they feel, and when they talk about their experiences. Ask questions, and keep an open mind
• Help where you can. There is always more work to be done in this space, so get involved and do some of it!

And that really is it. If you sincerely care about the people in your team, genuinely listen to what they have to say and then do what is in your power to help them, you are already an ally – so thank you! The rest of this post will dive into these three topics a bit deeper in the context of supporting women, but note that they can also be applied more broadly, too.

Care

“Do you really care?” can feel like a trivial question – if you are already reading this article you must, right?! But, in the spirit of being intentionally provocative – do you really  care? To quote a quick Google search, to care is to “apply serious attention or consideration.” Skimming headlines and liking LinkedIn posts about diversity is OK, but if you have given serious attention to gender diversity in your team, do you roughly know what the proportional gender split is? And do you have an opinion on what the biggest changes your company could make to foster inclusivity are?

An interesting (albeit dated) piece of research from Glassdoor shows that the majority of respondents to a survey identified diversity as an important factor when evaluating employers. It also found that the majority also think their current companies should be doing more to increase diversity. So, on the surface at least, it seems like people do care!

However, the same survey found that the majority of candidates did not know of, or were unsure of, any diversity initiatives in place at their current companies. So, while most people care, they haven’t even looked at what is already being done at their current workplace. Personally, I don’t think this result is surprising – there is a strong moral argument for diversity and inclusion that is easy for most people to emotionally connect with. But caring about D&I in a business – without even finding out what policies are in place to support it – isn’t so different to caring about a football team without knowing any of the players’ names.

If you care, you should also be open to changing your opinion on topics as you learn more. As an example, I strongly dislike hierarchical work environments. So much so, in fact, that I can often lean towards being anti-hierarchy. Specific to the topic at hand, strongly hierarchical spaces can result in women and other groups being disadvantaged, as they are subject to layers of management who do not understand (or possibly do not respect) them. 

In an extremely hierarchical space, layers of hierarchy will only have close interactions with their own level and the levels immediately above/below, meaning biases – even if unconscious – can result in sections of hierarchy becoming all but inaccessible to women (the glass ceiling). But, actually, an egalitarian approach to things doesn’t work when people are not treated as equals, or when people don’t act equally.

Imagine your best decision maker (regardless of gender) is extremely quiet, non-assertive, or introverted. Just because they technically have an equal voice to the rest of their team, they might not get the space they need to use it. Hierarchy can add structure that ensures the right people have the authority to get the right information and make the right decisions for the company when they need to. 

While this example was non-malicious, imagine the challenges somebody who already does not feel included might go through. Hierarchy can let people ‘pull rank’ when required, and ensure they are heard when they need to be. Furthermore, humans are a social species, and informal hierarchies will inevitably arise in the absence of formal ones – but formal hierarchies are much easier to assess for inclusivity and fairness. I came to see this other side of hierarchy through listening to the lived experiences of female managers. So while I am still egalitarian at heart, listening to the experiences of others has helped me to understand and appreciate that there are both good and bad aspects to hierarchical structures that should be accounted for when trying to support inclusion and diversity. 

Below I have listed several questions that you might want to think about. You don’t need to know the answer to all of these questions, but if you want to support gender diversity and inclusivity in your team, you should know some, and be interested in finding out more about some of the others. 

• What is the proportional gender split in your industry?
• What is the proportional gender split in your team?
• What is the proportional gender split at each stage of your recruitment funnel?
• Does your company have diversity targets and, if so, what are they?
• Do you have women in leadership roles? If not, why not? Role models are important!
• How inclusive do you think your company’s culture is?
• What are the biggest things you think should be changed to make it more inclusive?
• What sorts of things does your company celebrate, and are they inclusive?
• How often do you hear a meeting moderator invite a non-assertive speaker to contribute to a meeting?
• Is there flexibility around office hours available for people with caring responsibilities?
• How good is your company’s maternity leave policy?
• How accessible is your office to pregnant women?
• Have you considered whether your job descriptions use language that women are less likely to engage with?
• What companies do gender diversity well, and what can you learn from them?
• Who are the experts in this domain, and what are they saying?

In contrast, here is a list of things to avoid if you care:

• Patting yourself on the back for slacktivism
• Turning other peoples’ struggles into a way to self-promote
• Not engaging with an issue because it doesn’t affect you personally
• Not engaging with an issue because it makes you uncomfortable
• Thinking of diversity issues as dehumanized ‘group’ issues, as opposed to issues that affect real individuals that you work with

Listen

After caring comes listening. You might be surprised at how many people care about gender diversity and inclusivity, and can quote all the stats, but don’t listen to the lived experiences of those around them. Listening should be easy – give people space to talk, and believe what they say. There is no need to be dismissive – every person and every company is different, so every set of experiences will be different.

Maybe a female coworker had a negative interaction and they think gender bias is the cause – whereas you think it was probably not gender-related at all. The truth is, from a diversity and inclusivity perspective, it doesn’t really  matter what the cause was. For some reason, quite possibly related to a long history of broader social interactions that have had negatively-gendered undertones, your colleague doesn’t feel included. They are now more likely to leave your company, and the industry altogether. Everybody loses. Listen to what people say, how they feel, and support them as individuals.  Don’t shut them down, or debate. You don’t need to be an expert about what every feminist issue is. Just listen with an open mind to the specific issues that the individual in front of you has experienced. Assume the amazing people you are working with know what they are talking about and are telling the truth.

Note that listening and supporting individuals  and groups of individuals does not mean you need to agree with every broad topic! We often discuss recruitment diversity at Peak, and sometimes it gets raised that well-known statistics indicate unconscious biases against women that can affect them during recruitment. As a Decision Intelligence company, we monitor our conversion rates at each recruitment stage, disaggregated by gender (important!), and have found no evidence to suggest that –at Peak – women are disadvantaged. While it is true that general society might have an unconscious bias against women in tech roles, it is also possible that this bias doesn’t exist in certain individuals, or at certain companies. Given we have data, the existence and influence of unconscious bias is something that can be respectfully discussed and debated, and even disagreed on.

What can’t be debated? We have had female applicants comment that their candidature experience was all-male, and that this made them feel uncomfortable. That was their experience. For context, we have a three-round interview process in the Peak data science team, and over those three rounds a candidate might meet seven Peak-ers – an all-male process is rare and doesn’t reflect the diversity within the Peak team. But dismissing these people as unlucky and pointing to these stats doesn’t help our female candidates feel more included. Instead, we have put a couple of simple measures in place to ensure mixed panels, particularly for diverse candidates. It’s a simple fix that makes everybody happier and better reflects Peak to our candidates. All from listening.

Below I have prepared a shorter list of questions around listening that you might want to think about. Again, you don’t need to know all the answers – consider them a starting point for trying to listen more. 

• When did you last ask a woman in your team about her experiences with inclusivity?
• What do the women in your team consider to be the top barriers to inclusivity at your company?
• What are the top things women in your company would like to change?
• Is there a committee or an internal group where people discuss issues and plan change?

And again, here is a list of things to avoid in order to listen better:

• Dismissing people’s personal experiences
• Debating people’s personal experiences
• Pointing out how other people might face similar challenges
• Using the conversation as a way to make people listen to you

Help

With the above foundations in place, the most impactful thing you can do is help. Helping is the action that actually makes things better. Any help is better than none, and not everybody needs to help in the same way. In fact, it is usually better when people use their unique skills or positions to help in ways that others can’t. There are countless ways to directly help, and there are a lot of great resources with ideas for getting directly involved. To name a few, you can:

• Join a Diversity and Inclusion committee
• Review your recruitment processes
• Advocate for diversity OKRs
• Mentor or help people from different backgrounds to your own get into the industry
• Start a reading group
• Get involved in wider communities

There are other ways you can help, too, that you might not even realize. For example, the data science team at Peak have been working on a more detailed Part-time/Flexible Work Policy lately – did you know that publicly-available stats suggest that 38% of women work part-time compared to just 13% of men? Flexibility helps people with caring responsibilities, which are traditionally more likely to be taken on by women.

Coming up with a flexible work policy isn’t easy, though; we have several different roles in the team that are functionally different, and would need to support flexibility in different ways. We need to consider what sort of part-time work is feasible for each role, and how that would impact the business. We also need to consider if job sharing is a possibility. But, anybody can help with this. You don’t need to be a woman, or a carer, or even want to work part-time to help review a policy or come up with some ways to make it viable. It’s a curious observation that people will readily volunteer to help with tasks that ultimately do not affect them like debugging, testing some code, or even moving some boxes, but feel like it is inappropriate to help on a diversity initiative because it does not affect them.

Another example of a different way to help involves the maternity package at Peak. As a fast-growing startup, our team grew faster than some of our policies did. The Peak maternity package was one such example, and people would discuss it with concern from time to time, but with little progress. When one of the managers in the team (for whom a maternity package would not benefit) heard this, they reviewed what had been done so far, gave feedback on how to restructure the proposal, and helped bring urgency to the issue. The proposal was a success and now Peak has a policy in place that is competitive for a business our size. The manager in this example didn’t need to own the proposal, or write the proposal, or come up with any of the ideas – the impact they had came from listening to the concerns of the team, then using their experience and position to help remove some of the blockers.

A final simple example of helping in small ways is to encourage women around you to take the opportunities that you know they would be awesome at, and to encourage equal participation. It is known that men can feel more confident with less experience than women, which is something that we have seen in our own data. Also, studies have shown that women are less likely to consider themselves for jobs that men might, based on their perception of required experience. This can lead to women taking longer to move into senior roles, because they might not feel like they are ready to apply when the opportunity arises, and the position might go to a less-qualified person who is more confident in their own abilities. If you think a person is a superstar, or can handle more responsibility, tell them! Encourage them! And, if you are moderating a meeting and you see certain women might not be getting the space they need to contribute, call it out and give them that space.

So, you see – helping is something that anybody can do, regardless of their background. You just need to care and listen in order to understand how to help best. Only one question to round out the section this time…

• What was the last thing you did to help?

And a few what-not-to-do’s on helping:

• Don’t boast about the things you have helped with. Instead of “I am on the diversity committee and I led this project to success,” say: “The diversity committee has achieved this great success.” Celebrate the outcome, or the involvement of the people you are trying to help. Helping others shouldn’t be about you!
• Don’t think you have to ‘fix’ minorities. Every workplace has a default persona. Work on optimizing that default (and making it more inclusive), rather than trying to change people into something that, by the definition I have conveniently taken, is sub-optimal. A classic example of this relates to women’s confidence. If women in an organization have lower confidence in their skills than men, the ‘problem’ might actually be that the men are too rash or demonstrate a lack of self awareness. If women are less likely to contribute in a meeting, it might not have anything to do with confidence (or assertiveness) – I am sure we have all attended poorly-moderated meetings where a couple of over-talkative individuals prevent anybody else from contributing…then the last 20 minutes becomes a chat about rugby!
• Don’t feel cheated because you helped on something but nobody helped you back. 
• Don’t expect minority groups to do all the hard work for you – Part 1. When the new maternity leave policy was released at Peak, a number of people asked when a new paternity policy would be released. To be fair, our People team did pick this up pretty quickly, but there was nothing stopping people who wanted a better paternity policy getting involved to help make that change themselves. At best it can feel inconsiderate – “what they have achieved is ok, but it would have been better if it benefited me more.” At worst, it can feel like a sense of entitlement – “it isn’t fair that I miss out, they should work hard to improve my situation next!” In fact, I have seen this “where’s mine?” attitude completely block great ideas in other workplaces. Don’t be like that!
• Don’t expect minority groups to do all the hard work for you – Part 2. When minority groups have to disproportionately work on company-wide initiatives to foster diversity and inclusivity, they have disproportionately less time to work on their actual work. Consider the earlier example about diverse interview panels; because the pool of female interviewers is smaller, it could be easy to end up in a situation where women spend a lot more of their working week (on average) in interviews than men. An important part of being an ally is taking some of that burden off of minority groups. To quote Amy, our Head of Data Science Operations, “it shouldn’t be a trade off between being more successful in your career vs. being a martyr for the cause, but sometimes it can feel that way.”

Conclusion

Allyship is incredibly important if we want to see real change in the industries within which we work. Implementing change is hard enough without having to drive it from a minority position, through a hierarchy that might not understand you, using processes and structures that were not designed for you, when you already don’t feel included. 

Supporting diversity and inclusion as an ally shouldn’t be hard, though, and can be reduced to the three core ingredients of caring, listening and helping. While the focus of this piece has been on gender diversity, these principles can equally be applied to other demographics.

Thank you for reading this – if you have any questions or points to discuss, feel free to reach out.

On a more social front, we know that people don’t always make great decisions, and we know that people are not always good at justifying the decisions they make. In fact, people disagree with each other all the time!

So, where do we as a society draw the line and say “OK AI, I don’t agree with you, but you have adequately explained how you reached that conclusion, and I trust the decision you have made.”

Because the field is moving so fast, the willingness for someone to trust what an AI system outputs is still quite a personal thing. At Peak, we’ve previously found that, while one person might be happy to accept AI into their workflow without explainability so long as its performance is demonstrably strong (perhaps through an A/B test), another person in the same role might want a full break down of every output, with plots and a plain English summary.

I expect that as society uses AI more, and becomes more accepting of it, the focus of Explainable AI will shift, becoming less “why should a less technical person believe this?” and more “show me that you aren’t being biased or unethical.”

On the above: one understated property of Explainable AI concepts is their ability to find biases in what we currently do without AI.

Explainable AI can act like the old fashioned canary in the coal mine. If an Explainable AI model says, “do not hire this person because they are a woman,” it will usually be because it learned that the underlying processes themselves are biased against women. Now, some companies seem to blame the canary for its silence – but that is another issue altogether!