Employer will accept a Masters degree in Computer Science, Engineering, or related field and 3 years of work experience in the job offered or in a Software Engineer related occupation. Position requiresbr br 1. Creating datalake in AWS cloud using EMR, S3, Spark, Hive, Cloudformation, Redshift, Service Catalog, Scala, Python, Java, and Lambda;br 2. Building data pipelines using spark, Tidal, S3, Redshift, Scala, Python, Java, and Lambda;br 3. Creating Continuous IntegrationContinuous Deployment pipelines using Devops tools including Jenkins, Terraform, Github, and Unix bash script;br 4. Creating and automating datalake governance policy and access control using Apache Hive, Hive Metastore, HDFS, and Hive Server2;br 5. Creating and managing data connectivity interface to datalake using Spark Thrift, Hive, Beeline, Kerberos and Livy services;br 6. Migrating data pipelines from Vertica and Alteryx to EMR using Spark, Hive, Scala, Python, Java, bash script, Jenkins, cloudformation, Lambda, Github, Vertica, Alteryx, and Tidal;br 7. Creating tools to monitor and optimize AWS cost by using AWS Cost Explorer, Trusted Advisor, and recommendations services;br 8. Creating reports to monitor ETL jobs, job metrics, job performance using QlickSense, PowerBI, Tableau, AWS Athena, Tidal, and AWS Lambda; andbr 9. Creating hive UDF using Java, Python and Scala to create custom functions.
Categories: eb3
0 Comments