Job Summary

The skilled AI Ops Model Monitoring professional to monitor the post-deployment health, performance, and reliability of AI/ML models — including traditional ML and Generative AI/LLM systems. Build and maintain advanced monitoring frameworks to detect drift, degradation, anomalies, and GenAI-specific issues (e.g., query-context-relevance and end user satisfaction). Leverage Databricks native capabilities (Inference Tables, Unity Catalog, Model Serving) alongside broader MLOps tools to ensure scalable, automated observability and proactive governance

•    Implement a GenAI evaluation framework to continuously measure output quality against trusted sources and drive model improvement and governance.  
•    Design and implement continuous monitoring pipelines for ML and GenAI models, tracking core metrics (accuracy, precision/recall, latency, drift) and GenAI-specific signals (query-context-relevance and end user satisfaction in real-time or batch modes.
•    Set up automated alerting, dashboards and incident workflows for model/endpoint health degradation.
•    Leverage Databricks Inference Tables to capture and analyse serving logs (inputs, predictions, timestamps) from Mosaic AI Model Serving endpoints; build custom processing jobs to derive profile/drift metrics, anomaly detection, and performance trends.
•    Build GenAI observability: monitor query → context → relevance chains in RAG pipelines, track embedding similarity, retrieval quality, and end-to-end response metrics using logged inference data.
•    Collaborate with data scientists & AI engineers to define monitoring thresholds, baselines, drift detectors, and automated retraining/rollback triggers.
•    Manage model lifecycle governance (versioning, compliance, auditability) using Databricks Unity Catalogue, MLflow, and related tools.
•    Contribute to enhancing the organization’s MLOps platform with Databricks-centric monitoring features (e.g., scheduled jobs on inference tables) for scalability and ease of use.

Key Responsibilities

Strategy
•    Implementation of continuous Model Monitoring for all use cases across CIB

Business
•    Understand the Business requirement and execute the ML solutioning and ensue the delivery commitments are delivered on time and schedule.

Processes
•    RAI, Safety and Security
•    Model Validation | Model Monitoring and Improvements
•    Stakeholder Management
•    Risk Management
 
Risk Management
•    Ownership of the delivery, highlighting various risks on a timely manner to the stakeholders.
•    Identifying proper remediation plan for the risks with proper risk roadmap.

Governance
•    Awareness and understanding of the regulatory framework, in which the Group operates, and the regulatory requirements and expectations relevant to the role.

Regulatory & Business Conduct
•    Display exemplary conduct and live by the Group’s Values and Code of Conduct. 
•    Take personal responsibility for embedding the highest standards of ethics, including regulatory and business conduct, across Standard Chartered Bank. This includes understanding and ensuring compliance with, in letter and spirit, all applicable laws, regulations, guidelines and the Group Code of Conduct.
•    Effectively and collaboratively identify, escalate, mitigate and resolve risk, conduct and compliance matters.

Key stakeholders
•    Business Stakeholders
•    AIML Engineering Team 
•    AIML Product Team 
•    Product Enablement Team
•    SCB Infrastructure Team
•    Interfacing Program Team

Skills and Experience

•    Use NLP, Vision and ML / Gen AI / LLM techniques to bring order to structed and unstructured data
•    Work within the Engineering Team to design, code, train, test, deploy and iterate on enterprise scale machine learning systems
•    Experience in Model Monitoring and Improvements for live AI systems
•    Work alongside an excellent, cross-functional team across Engineering, Product and Design
•    create solutions and try various algorithms to solve the problem.
•    Stakeholder Management
•    Risk Management 

Qualifications

•    Bachelor’s or Master’s in Computer Science, Data Science, AI/ML, Engineering, or equivalent practical experience.
•    3–7+ years in MLOps, production ML engineering, AI reliability, or AIOps roles with hands-on model monitoring experience.
•    Strong proficiency in Databricks for model monitoring: Inference Tables for logging and analyzing model serving requests/responses, Unity Catalog integration, Model Serving endpoints, and building monitoring dashboards/jobs/queries on captured logs.
•    Experience implementing query, context, and relevance monitoring for LLMs/GenAI (e.g., RAG relevance scoring, context faithfulness, retrieval-augmented metrics) using inference log data.
•    Hands-on with Python and ML frameworks (PyTorch, TensorFlow, scikit-learn); familiarity with LangChain/LlamaIndex or similar for GenAI pipelines is a plus.
•    Expertise in drift detection, performance tracking, and observability tools (Evidently AI, Alibi Detect, WhyLabs, Arize, Fiddler, Prometheus/Grafana applied to ML).
•    Proficiency building ML CI/CD pipelines (MLflow, Databricks Workflows, Airflow, Kubeflow).
•    Solid cloud experience, especially Databricks (Model Serving, Mosaic AI endpoints), plus AWS/Azure/GCP ML services.
•    Understanding of containerization/orchestration (Docker, Kubernetes) and observability stacks for production AI workloads.
•    Strong analytical troubleshooting for production ML/GenAI issues.
•    Excellent collaboration and communication skills.
•    Direct experience monitoring production LLMs/GenAI at scale (hallucination detection, prompt drift, safety/guardrails) using Databricks inference logging
•    Familiarity with responsible AI tools (bias/fairness in AIF360,Fairlearn) and GenAI specific observability (e.g Phoenix , Langsmith)

About Standard Chartered

We're an international bank, nimble enough to act, big enough for impact. For more than 170 years, we've worked to make a positive difference for our clients, communities, and each other. We question the status quo, love a challenge and enjoy finding new opportunities to grow and do better than before. If you're looking for a career with purpose and you want to work for a bank making a difference, we want to hear from you. You can count on us to celebrate your unique talents and we can't wait to see the talents you can bring us.

Our purpose, to drive commerce and prosperity through our unique diversity, together with our brand promise, to be here for good are achieved by how we each live our valued behaviours. When you work with us, you'll see how we value difference and advocate inclusion.

Together we:

• Do the right thing and are assertive, challenge one another, and live with integrity, while putting the client at the heart of what we do
• Never settle, continuously striving to improve and innovate, keeping things simple and learning from doing well, and not so well
• Are better together, we can be ourselves, be inclusive, see more good in others, and work collectively to build for the long term

What we offer

In line with our Fair Pay Charter, we offer a competitive salary and benefits to support your mental, physical, financial and social wellbeing.

• Core bank funding for retirement savings, medical and life insurance, with flexible and voluntary benefits available in some locations.
• Time-off including annual leave, parental/maternity (20 weeks), sabbatical (12 months maximum) and volunteering leave (3 days), along with minimum global standards for annual and public holiday, which is combined to 30 days minimum.
• Flexible working options based around home and office locations, with flexible working patterns.
• Proactive wellbeing support through Unmind, a market-leading digital wellbeing platform, development courses for resilience and other human skills, global Employee Assistance Programme, sick leave, mental health first-aiders and all sorts of self-help toolkits
• A continuous learning culture to support your growth, with opportunities to reskill and upskill and access to physical, virtual and digital learning.
• Being part of an inclusive and values driven organisation, one that embraces and celebrates our unique diversity, across our teams, business functions and geographies - everyone feels respected and can realise their full potential.