3 years of experience Utilizing AWS cloud for the collection, storage and processing of raw data from HDFS, onprem, relational databases and external APIs into AWS by leveraging AWS tools like S3, EMR, Athena, Glue Lambda, and Redshift; extracting, loading and transforming data in the Redshift cloud Datawarehouse and developing mechanisms to provision the data to business users, data scientists, BI Tools, and downstream systems for data analysis; creating Lambda functions using Python to process files in realtime in response to events trigged from S3, SQS, SNS and other Lambda functions; developing Glue scripts to transform andor process Batch data using Spark and utilizing Glue Data Catalog to store and access S3 object metadata; developing Spark scripts to perform complex transformations in AWS EMR involving large datasets by considering cost and performance; creating and maintaining EC2 instances to install and run applications like Looker, Zeppelin and execute various scripts; using Python as the programming language to create ETLdata pipelines and run applications in AWS Lambda, EMR, EC2 and Glue; and using BI Tools like Looker or Tableau to create and publish dashboards and graphs. Employee reports to LexisNexis USA office in Raleigh, NC but may telecommute from any location within the U.S. Experience can be concurrent.
Categories: eb3
0 Comments