In this chapter, we started introducing or rather mapping technologies into the various data lake layers. In this chapter, we started with the technology introduction in the data acquisition layer. We started the chapter with the layer definition first, and then listed down reasons for choosing Sqoop by detailing both its advantages and disadvantages. We then covered Sqoop and its architecture in detail. While doing so, we covered two important versions of Sqoop, namely version 1 and 2. Soon after this theoretical section, we delved deep into the actual workings of Sqoop by giving the actual setup required to run Sqoop, and then delved deep into our SCV use case and what we are achieving using Sqoop.

After reading this chapter, you should have a clear understanding of the data acquisition layer in our data lake architecture. You should have also gotten in-depth details on Apache Sqoop and what are the reasons for choosing this as a technology of choice for implementation. You would also have gained knowledge on the actual working of Sqoop and how we can use it in action by going through the working example. You would by now have also implemented data acquisition functionality from PostgreSQL to HDFS using Sqoop, which kickstarts our SCV use case using Data Lake.

In the next chapter, we will move on to the next layer in our Data Lake, namely the Messaging Layer, and introduce the technology choices that we have chosen in detail, similar to what we have done in this chapter.