Web Scraping Using Python

What is Web Scraping

Web scraping is a method to extract data from websites. Using web scarping we can store unstructured website data to structured website data. There are different ways of scrape websites. In this, we’ll do web scraping with python.

In general, web data extraction is used by people and businesses who want to make use of the vast amount of publicly available web data to make smarter decisions.

Requirement for Web Scraping

  1. Selenium
  2. Beautiful Soup
  3. Request
  4. lxml

Installation

We have to install following libraries and packages for web scraping and for that wehave to run following commands in python editor.

Import Libraries

We have to import following libraries.

Website URL

Webdriver

As we all know, to get the content from the website we need to provide a web driver of the browser we are using.

Fetching data from website

First we have to create empty list for getting data from website

Now, you have to run a loop and fetch all data and append it to the list. To fetch the data we first need to inspect the webpage and then we need to check the class name of that particular div tag and we need to write it in the code.

Data Frame

Convert data into CSV file

df.to_csv(‘products.csv’, index=False, encoding=’utf-8')

Student at Charusat B.tech IT