Azure Data Engineer - Assistant Manager

Description - Bachelor’s or higher degree in Computer Science or a related discipline; or equivalent (minimum 7 years work experience). - At least 4+ years of consulting or client service delivery experience on Azure - At least 4+ years of experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases such as SQL server and data warehouse solutions such as Azure Synapse - Extensive experience providing practical direction with using Azure Native services. - Extensive hands-on experience implementing data ingestion, ETL and data processing using Azure services: ADLS, Azure Data Factory, Azure Functions, Azure Logic App Synapse/DW, Azure SQL DB, Event Hub, IOT Hub, Azure Stream Analytics, Azure Analysis Service, Databricks, Cosmo DB etc. - Minimum of 4+ years of hands-on experience in Azure and Big Data technologies such as Java, Python, SQL, ADLS/Blob, PySpark and SparkSQL, Databricks, HD Insight and live streaming technologies such as EventHub, Azure Stream Analytics etc. - Well versed in DevSecOps and CI/CD deployments - Cloud migration methodologies and processes including tools like Azure Data Factory, Data Migration Service, etc. - Minimum of 4+ years of RDBMS experience - Experience in using Big Data File Formats and compression techniques. - Experience working with Developer tools such as Azure DevOps, Visual Studio Team Server, Git, Jenkins, etc. - Experience with private and public cloud architectures, pros/cons, and migration considerations. Responsibilities A Databricks Data Engineer is responsible for designing, building, and maintaining data solutions that leverage Databricks' powerful capabilities on the cloud platform. Key responsibilities include: - Designing and deploying data pipelines that efficiently process vast amounts of data using Databricks, optimizing for performance and scalability. - Collaborating with data scientists and business stakeholders to understand requirements and provide data solutions tailored to those needs. - Implementing robust data models and architectures utilizing Databricks, ensuring data quality and integrity. - Monitoring and troubleshooting data pipelines, proactively identifying and resolving issues to ensure smooth data flows. - Ensuring data security, governance, and privacy measures are in place, adhering to industry best practices. - Optimizing data storage and processing costs while maintaining high performance. - Engaging in continuous improvement activities, researching new technologies and methodologies to enhance data solutions. Qualifications - Bachelor’s or higher degree in Computer Science or a related discipline; or equivalent (minimum 7 years work experience). - At least 4+ years of consulting or client service delivery experience on Azure - At least 4+ years of experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases such as SQL server and data warehouse solutions such as Azure Synapse - Extensive experience providing practical direction with using Azure Native services. - Extensive hands-on experience implementing data ingestion, ETL and data processing using Azure services: ADLS, Azure Data Factory, Azure Functions, Azure Logic App Synapse/DW, Azure SQL DB, Event Hub, IOT Hub, Azure Stream Analytics, Azure Analysis Service, Databricks, Cosmo DB etc. - Minimum of 4+ years of hands-on experience in Azure and Big Data technologies such as Java, Python, SQL, ADLS/Blob, PySpark and SparkSQL, Databricks, HD Insight and live streaming technologies such as EventHub, Azure Stream Analytics etc. - Well versed in DevSecOps and CI/CD deployments - Cloud migration methodologies and processes including tools like Azure Data Factory, Data Migration Service, etc. - Minimum of 4+ years of RDBMS experience - Experience in using Big Data File Formats and compression techniques. - Experience working with Developer tools such as Azure DevOps, Visual Studio Team Server, Git, Jenkins, etc. - Experience with private and public cloud architectures, pros/cons, and migration considerations.