We are seeking an experienced Data Engineer with 3+ years of hands-on expertise in PySpark, Python, and Boto3. The candidate will be responsible for creating data pipelines, building data assets, and optimizing code using Spark SQL and PySpark. Knowledge of AWS architecture, particularly S3, EC2, Lambda, Redshift, and CloudFormation, is required, along with experience in version control systems like Git and JFrog Artifactory. The role also involves code refactoring of legacy codebases, implementing unit tests, and resolving complex bugs to improve performance and maintainability.

• Design and develop robust data pipelines and assets using advanced data engineering techniques.
• Utilize PySpark, Python, and Boto3 for efficient data processing, adhering to Python best practices.
• Optimize performance and code efficiency with Spark SQL and PySpark.
• Manage version control and code repositories with Git and JFrog Artifactory.
• Architect and deploy solutions on AWS, including S3, EC2, Lambda, Redshift, and CloudFormation, explaining the benefits of each.
• Refactor legacy code to enhance readability, maintainability, and performance.
• Implement unit tests and follow test-driven development (TDD) to ensure code quality.
• Troubleshoot and debug complex issues in performance, concurrency, or logic flaws in code.