Talascend is currently seeking a Azure Databricks Engineer for a contract opportunity with our client in Glendale, Arizona.
Overview
This role involves designing, implementing, and maintaining scalable enterprise data platform solutions using Azure Databricks and related Azure services. The engineer will collaborate with data teams to support batch, streaming, ELT/ETL, and real-time analytics workloads, ensuring platform security, governance, and performance optimization.
Responsibilities
- Collaborate with data platform and integration teams to architect and implement enterprise data solutions utilizing Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2, and related services.
- Design and develop data engineering frameworks supporting various data workloads and analytics needs.
- Administer and maintain Azure Databricks workspaces, including cluster configuration, job scheduling, security, and workspace governance.
- Implement enterprise security architecture, including identity management, RBAC, PIM, data encryption, private networking, and secure access patterns.
- Establish enterprise data governance standards with Unity Catalog, emphasizing data lineage, quality, and metadata management.
- Optimize workloads through performance tuning, Spark optimization, caching, partitioning, and cost management.
- Develop monitoring, logging, and operational dashboards using Azure Monitor, Log Analytics, and other tools.
- Create reusable architecture patterns, standards, and operational procedures for data platform implementation.
- Support migration efforts from legacy systems to Azure Databricks and modern Lakehouse architecture.
- Provide technical support and troubleshoot production incidents related to Azure Databricks and data pipelines.
Qualifications
- At least 8-10 years of experience in designing and implementing enterprise data platforms.
- At least 5 years of hands-on experience with Azure Databricks in enterprise environments.
- Experience deploying modern Lakehouse solutions with Delta Lake and Azure Databricks.
- Proficiency in administering Azure Databricks workspaces, clusters, Unity Catalog, and security configurations.
- Strong skills in developing data engineering solutions using Apache Spark, PySpark, and Spark SQL.
- Experience with enterprise data governance, metadata management, data lineage, and access controls.
- Knowledge of enterprise security architectures, including RBAC, Managed Identities, and Key Vault integration.
- Experience implementing data quality frameworks, monitoring, and observability tools.
- Strong understanding of data modeling techniques such as dimensional modeling, Data Vault, and medallion architecture.
- Familiarity with Azure Data Lake Storage Gen2, Azure Key Vault, Azure Monitor, Microsoft Entra ID, and Azure networking.
- Proficiency in SQL programming, Python, and PySpark development.
- Experience implementing CI/CD pipelines with Azure DevOps, testing, and deployment automation.
Preferred Qualifications
- Experience supporting migration projects from legacy data warehouses to Lakehouse architectures.
- Knowledge of Azure networking, private endpoints, and security best practices.
- Strong organizational and communication skills for collaboration across teams.
We thank all applicants for their interest. However, only those qualified individuals who closely meet the qualifications of the position will be contacted. The details of the position are only a summary, other duties may be assigned as necessary.
Background Check and Drug Screen may be required.
Talascend is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.
Military connected talent encouraged to apply.