Ampcus Inc. is a certified global provider of a broad range of Technology and Business consulting services. We are in search of a highly motivated candidate to join our talented Team.
Job Title: Senior Cloud Engineer - Azure Infrastructure & Databricks Platform
Location(s): Warren, MI
(Remote)
Position Summary
We are seeking a Senior Cloud Engineer to design, implement, and administer the Azure infrastructure supporting an enterprise data and AI platform. This is a cloud infrastructure and platform engineering role focused on Azure Databricks, ADLS Gen2, Azure Data Factory, and Azure Machine Learning.
The Senior Cloud Engineer will own the platform foundation, including infrastructure as code, network and identity architecture, compute governance, observability, reliability, and cloud cost management. The role will work closely with Databricks administration, data engineering, and data science teams to establish platform standards, reference architectures, and secure self-service capabilities.
The ideal candidate combines deep Azure infrastructure expertise with strong hands-on Databricks administration experience and can serve as a senior technical authority for complex platform and architecture decisions.
Key Responsibilities
Azure Infrastructure & Platform Engineering
Design, implement, and administer the Azure foundation supporting the enterprise data and AI platform.
Develop and maintain reusable Terraform modules for Azure Databricks, ADLS Gen2, Azure Data Factory, Key Vault, networking, and Azure Machine Learning.
Manage Terraform remote state, module versioning, drift detection, and CI/CD deployment pipelines.
Establish and maintain development, test, and production environment promotion processes.
Design secure Azure networking architectures, including private endpoints, VNets, hub-and-spoke topology, VNet injection, NSGs, firewall rules, and data exfiltration controls.
Implement Entra ID, managed identities, RBAC, Key Vault, and secure identity/access patterns.
Define enterprise ADLS Gen2 architecture, including storage accounts, hierarchical namespaces, containers, ACLs, lifecycle policies, tiering, encryption, and key management.
Design access patterns between ADLS Gen2, Databricks, and downstream consumers.
Azure Data Factory
Own Azure Data Factory from the platform infrastructure perspective.
Provision and manage managed and self-hosted integration runtimes.
Configure managed VNet and secure connectivity.
Design linked-service credentials and managed identity integrations.
Support environment promotion, deployment automation, monitoring, and platform reliability.
Partner with data engineering teams; production ETL/ELT pipeline development remains outside this role.
Azure Machine Learning & AI Infrastructure
Provision and operate Azure Machine Learning workspaces and compute clusters.
Support GPU quota management, capacity planning, and workload infrastructure.
Support MLflow model registry integration and model-serving endpoints.
Establish secure identity and networking between Azure ML and Unity Catalog.
Ensure AI/ML infrastructure follows enterprise security, governance, and operational standards.
Observability & Reliability
Build and maintain monitoring and observability across the Azure platform.
Configure Azure Monitor, Log Analytics, diagnostic settings, dashboards, and alerting.
Develop operational runbooks and support reliability engineering practices.
Troubleshoot complex infrastructure, connectivity, authentication, and platform issues.
FinOps & Capacity Management
Establish cloud cost visibility and optimization practices across Databricks DBU and Azure storage consumption.
Implement tagging, chargeback/showback, budget alerts, and cost allocation strategies.
Evaluate reserved capacity and other optimization opportunities.
Perform capacity planning and forecasting for platform growth.
Analyze Databricks usage and identify opportunities to reduce unnecessary consumption.
Databricks Platform Administration
The role partners with a dedicated Databricks administration team and provides the architecture, standards, reference configurations, and senior technical guidance supporting the platform.
Account & Workspace Administration
Configure Databricks account-level settings and workspace environments.
Define workspace provisioning standards and topology strategies.
Establish workspace-level configuration and administrative delegation.
Contribute to multi-workspace architecture and governance.
Unity Catalog
Design and govern Unity Catalog metastores and regional strategies.
Establish catalog, schema, table, and permission models.
Configure storage credentials and external locations.
Support Hive Metastore migrations.
Implement lineage, auditing, and Delta Sharing configurations.
Identity & Access Management
Integrate Databricks with Microsoft Entra ID.
Implement SCIM provisioning, identity federation, groups, and entitlements.
Manage service principals and token policies.
Establish secure authentication and authorization patterns.
Compute Governance
Develop and enforce Databricks cluster policies.
Establish instance pool, node type, and runtime standards.
Configure autoscaling and autotermination policies.
Evaluate Photon and serverless compute options.
Configure and size SQL warehouses based on workload requirements.
Cost & Usage Management
Use Databricks system tables to analyze platform consumption.
Establish usage attribution and DBU forecasting.
Identify major sources of Databricks spend.
Recommend and implement remediation strategies for inefficient resource utilization.
Technical... For full info follow application link.
Ampcus is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identify, national origin, age, protected veterans or individuals with disabilities.
