Posted about 1 year ago
Microsoft Platform Big Data Engineer
Toronto | Mississauga
6 Month Contract | Contract to hire
Are you a Toronto area based Azure Big Data Engineer or Developer who can provide on site consulting services to one of Pythian clients; including planning, designing and implementing new solutions using the latest Big Data technologies? Do you want to work to implement and help develop a cutting edge Big Data solutions, create data pipelines which will migrate data from customers on-prem systems and load it into a cloud hosted Enterprise Data Platform? Do you want to work on large scale custom Big Data consulting projects? If you have a passion for big data and solving complex problems then this could be the job for you!
What will you be doing?
- Working with the Data Architects to implement data pipelines
- Working with our Big Data Principal Architects in the development both proof of concepts and complete implementations.
- Working on complex and varied Big Data projects including tasks such as collecting, parsing, managing, analyzing, and visualizing very large datasets.
- Translating complex functional and technical requirements into detailed designs.
- Writing high-performance, reliable and maintainable code.
- Performing data processing requirements analysis.
- Performance tuning for batch and real-time data processing.
- Securing components of clients’ Big Data platforms.
- Diagnostics and troubleshooting of operational issues.
- Health-checks and configuration reviews.
- Data pipelines development – ingestion, transformation, cleansing.
- Data flow integration with external systems.
- Integration with data access tools and products.
- Assisting application developers and advising on efficient data access and manipulations.
- Defining and implementing efficient operational processes
What do we need from you?
While we realise you might not have everything on the list to be the successful candidate for the Microsoft Platform Big Data Developer job you will likely have at least 5 years experience in similar roles. The position requires specialized knowledge and experience in performing the following:
- Experience building data pipelines in Azure using Azure Data Factory
- Experience writing ETL (Any popular tools)
- Experience in data modeling, data design and persistence (e.g. warehousing, data marts, data lakes).
- Strong experience building solutions using Azure Databricks, Azure Analysis Services, Azure Kubernetes, Azure Functions and Azure Data Lake Storage Gen2.
- Strong Knowledge of Big Data architectures and distributed data processing frameworks: Hadoop, Spark, Kafka, Hive
- Experience and working knowledge of various development platforms, frameworks and languages such as Java, Python, Scala and SQL
- Experience with SQL Server and SSIS
- General knowledge of modern data-center and cloud infrastructure including server hardware, networking and storage.
- Strong written and verbal English communication skills
- Experience with BI platforms, reporting tools, data visualization products, ETL engines.
- Experience with data streaming frameworks.
- DevOps experience with a good understanding of continuous delivery and deployment patterns and tools (Jenkins, Artifactory, Maven, etc)
- Experience with Hbase.
- Experience in data management best practices, real-time and batch data integration, and data rationalization
Established in 1997, Pythian is a global IT Data company based in Ottawa, Canada that specializes in designing, implementing, and managing systems that directly contribute to revenue and business success. We help companies adopt disruptive technologies such as advanced analytics, big data, cloud, databases, DevOps and infrastructure management to advance innovation and increase agility. We are focusing on product development leveraging new and exciting technology that will empower our customers through digital transformation. More than just a job we hire people who love what they do! As a Microsoft Platform Big Data Engineer in Toronto, you will be part of our Customer Service Delivery team that is entrusted to manage our global client's mission critical systems as well as deploying cutting edge technology from blockchain to serverless and cloud databases, covering all modern data and infrastructure. They deliver first class personalized level of service to our clients across financial, educational, media, retail and many more.
Intrigued to see what a job is like at Pythian? Check us out @Pythian and #pythianlife.
Follow @PythianJobs on Twitter and @loveyourdata on Instagram!
Not the right job for you? Check out what other great jobs Pythian has open in the United States or Canada in general and around the world! Pythian Careers
- Pythian is an equal opportunity employer and welcomes applications from people with disabilities. Accommodations are available on request for candidates taking part in all aspects of the selection process.
- The successful applicant will need to fulfill the requirements necessary to obtain a background check.
- All applicants must be legally authorized to work in the United States of America or Canada permanently– Pythian will not sponsor, or file petitions of any kind on behalf of, a foreign worker to become a U.S or Canadian permanent resident based on a permanent job offer, or to otherwise obtain authorization to work in the U.S. or Canada
- No Recruitment agencies please