Lead / Senior / Data Engineer, 3-12 years, 20-37 LPA, Bangalore

From 5 to 10 year(s) of experience
₹ 25,00,000 - 37,50,000 P.A.

Job Description

Roles and Responsibilities

IDEAL CANDIDATE

- 3+ Years in Data Engineering (5+ and 7+ years for Sr and Lead roles)
- Worked on Airflow, AWS & EMR, Spark
- Should join within 30 days (preferably within 15 days)


ORGANISATION

Industry-leading Data Artificial Intelligence provider in the Retail industry. Their experience and capabilities enable global Retail leaders to enhance their services to their end users. They also enable customer vendors to streamline their operations and improve delivery metrics via Artificial Intelligence / Machine Learning. With deep experience in Data Science, Data Engineering, and Enterprise-Scale Storage and Compute. They help customers navigate the modern Cloud landscape effortlessly.


ROLE

  1. Build High level technical design both for Streaming and batch processing systems
  2. Design and build reusable components, frameworks and libraries at scale to support analytics data products
  3. Perform POCs on new technology, architecture patterns
  4. Design and implement product features in collaboration with business and Technology stakeholders
  5. Anticipate, identify and solve issues concerning data management to improve data quality
  6. Clean, prepare and optimize data at scale for ingestion and consumption
  7. Drive the implementation of new data management projects and re-structure of the current data architecture
  8. Implement complex automated workflows and routines using workflow scheduling tools
  9. Build continuous integration, test-driven development and production deployment frameworks
  10. Drive collaborative reviews of design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards
  11. Analyze and profile data for the purpose of designing scalable solutions
  12. Troubleshoot complex data issues and perform root cause analysis to proactively resolve product and operational issues
  13. Lead, Mentor and develop other Sr Data Engineers and Data engineers in adopting best practices and deliver data products.
  14. Partner closely with product management to understand business requirements, breakdown Epics,
  15. Partner with Engineering Managers to define technology roadmaps, align on design, architecture and enterprise strategy


CANDIDATE

Positions are Lead or Senior or Data Engineer

  1. Expert level expertise in building big data solutions
  2. Hands-on experience building cloud scalable, real time and high-performance data lake solutions using AWS, EMR, S3, Hive & Spark, Athena
  3. Hands-on experience in delivering batch and streaming jobs
  4. Expertise in an agile and iterative model
  5. Expert level expertise relational SQL
  6. Experience with scripting languages such as Shell, Python
  7. Experience with source control tools such as GitHub and related dev process
  8. Experience with workflow Scheduling tools like Airflow
  9. In-depth understanding of micro services architecture
  10. Strong understanding of developing complex data solutions
  11. Experience working on end-to-end solution design
  12. Able to lead others in solving complex Data and Analytics problems
  13. Strong understanding of data structures and algorithms
  14. Strong hands-on experience in solution and technical design
  15. Has a strong problem solving and analytical mindset
  16. Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders
  17. Able to quickly pick up new programming languages, technologies, and frameworks


LOCATION: Bangalore

SALARY : 20-37 LPA



Education

UG:B.Tech/B.E. in Any Specialization

Company Profile

Talent Bee Consulting Pvt. Ltd

An Industry-leading Data Artificial Intelligence provider in the Retail industry. Based in New York, you may have to do occasional travel to US
Company Info
X
View Contact Details+

Contact Company:Talent Bee Consulting Pvt. Ltd

Website:http://careers.talentbee.in

Salary:

25,00,000 - 37,50,000 P.A.

Functional Area:

Engineering - Software

Role Category:

Software Development

Role:

Technical Lead

Employment Type:

Full Time, Permanent