Introduction:

Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites using its HTML structure, In this post, I will explain basic fundaments of web scraping using python and also explore it by a live demonstration with two python libraries Beautifulsoup and requests respectively.

What you will learn from this post:

• basic understanding of web scraping
• how to extract data from a website using classes and HTML tags
• how to use requests module to get data
• how to use Beautifulsoup

Requirements:

• python3
• requests
• bs4

Installation:

• sudo apt-get python3-pip
• pip install requests
• pip install bs4

How to run this code

• there are two source code files, one is .py extention and another is .ipynb extention
• one can run Scraping with BeautifulSoup.py file in python by run this cammand in terminal "python Web Scraping with BeautifulSoup.py"
• one can run Scraping with BeautifulSoup.ipynb file in python /li>