Big Data Engineer will work for the building, maintaining and adding to our existing Big Data Architecture. Should be able to choose optimal open source solutions as per the requirement of projects.
- Suggesting and integrating any Big Data tools and frameworks required to fulfill project needs.
- Monitor and maintain the big data structure.
- Monitoring performance and advising any necessary infrastructure changes.
- Implementing data archival and data lake infrastructure.
- Implementing the ETL process whenever required.
Skills & Qualifications:
- Deep understanding of distributed computing systems.
- Management of Clusters of Kafka, Cassandra, Spark, etc in the on-premise data center.
- Proficiency with Cassandra and Spark
- Experience with integration of data from multiple data sources
- Knowledge of various ETL techniques
- Experience with Big Data open source ML tool-kits, such as SparkML, MLib, etc.
- Graduate in Science or Engineering stream with any certificate or diploma in Data Science / Data Engineering.