Databricks Developer

Expected

• Azure SQL

• T-SQL

• Apache Spark

• Python

Operating system

• The candidate should have experience in end-to-end Azure Cloud technologies, which may include but are not limited to:

• Storage Accounts

• Azure Databricks

• Azure Data Factory

• Logic Apps

• Azure Functions

• Azure Synapse

• Data Ingestion Using Azure Databricks and Apache Spark:

• The candidate should have a minimum of 3 years of experience building data ingestion pipelines using Azure Databricks and Apache Spark. This experience is crucial for efficiently processing and ingesting data into the Azure environment.

• T-SQL Proficiency:

• Strong proficiency in T-SQL is required, with at least 3 years of experience in Azure Synapse. T-SQL is important for querying and manipulating data within Azure Synapse.

• Python/Pyspark Expertise:

• The candidate should have extensive experience (3+ years) in Python and Pyspark for data processing and transformation tasks. These skills are valuable for working with data in various formats and performing necessary transformations.

• CI/CD Pipelines in Azure DevOps:

• A proven ability to build CI/CD (Continuous Integration/Continuous Deployment) pipelines in Azure DevOps is necessary. This skill is crucial for automating deployment processes, ensuring efficient and reliable deployments of data pipelines and applications in the Azure environment.