Senior Databricks Application Developer

Senior Analyst We are recruiting for a Databricks Application Developer in the Legal, Risk Management and Compliance (LRMC) team. Roles & responsibilities We are recruiting for a Databricks Application Developer in the Legal, Risk Management and Compliance (LRMC) team. • Develop business applications and AI-enabled solutions using Databricks, translating solution requirements into working applications and integrations. • Build Databricks applications and develop application logic, APIs, services, and integrations with enterprise systems. • Design and implement scalable batch and streaming data pipelines using Python, SQL, PySpark, and Delta Lake. • Apply Medallion Architecture principles across Bronze, Silver, and Gold layers to improve data quality, reusability, and consumption. • Create and manage Databricks Workflows, jobs, task dependencies, parameters, scheduling, monitoring, and failure handling. • Integrate governed data assets from Unity Catalog and apply appropriate access controls, lineage, and data governance practices. • Design database objects and data models, optimize SQL queries, and improve performance of data processing workloads. • Integrate Databricks applications with governed data, APIs, analytics services, and AI or machine learning capabilities. • Use GitHub for source control and implement CI/CD pipelines with GitHub Actions for automated testing and deployment. • Support unit testing, system integration testing, deployment, production readiness, troubleshooting, and technical documentation. Mandatory technical & functional skills • 3-5 years of experience in data engineering, data platform development, or Databricks application development. • Hands-on expertise with Azure Databricks, Python, PySpark, Spark SQL, SQL, Delta Lake, and notebook-based development. • Experience creating reliable ETL/ELT pipelines for batch and streaming data processing, including validation and error handling. • Strong understanding of Medallion Architecture, dimensional data modelling, database design, indexing, query optimization, and performance tuning. • Experience building Databricks applications, backend logic, REST APIs, and integrations with enterprise systems and data services. • Hands-on experience with Databricks Workflows, jobs, scheduling, task orchestration, monitoring, logging, and production support. • Experience with Unity Catalog, access controls, data lineage, governance, secrets, and secure data handling. • Proficiency with GitHub source control, branching, pull requests, code reviews, GitHub Actions, and CI/CD deployment pipelines. • Understanding of automated testing, Agile delivery, technical documentation, and operational readiness. Preferred technical & functional skills • Experience with Databricks Asset Bundles, Databricks SQL, Lakehouse architecture, and Infrastructure as Code. • Exposure to Azure services such as Azure Data Lake Storage, Key Vault, Data Factory, Entra ID, and Azure DevOps. • Experience integrating machine learning, generative AI, or governed AI services into Databricks applications. • Knowledge of Power BI or other analytics and data visualization platforms. • Understanding of risk management, compliance, governance, or other regulated business domains. Experience Level Senior Level