Gradient Descent

If you are familiar with maths, you might know that minimum or maximum value of a function is got by equating its derivative to zero. In other words, the value obtained when the slope(m) is zero considered as the min or max of that line(function).

 Gradient descent is one approach to find the minimum value of cost function. The algorithm goes through the θ values step by step, in direction of negative slopes and stops at slope zero.
The above image from hackernoon explains this concept well. (Please note, the parameter representation here is w not θ).

Gradient descent algorithm is given by,

θj :=θj −α∂θj ∂ J(θ0 ,θ1 )

α - Learning rate (Length of each step)
J(θ₀,θ₁) - Cost function
j - Iterator

The value of α plays a key role in determining the gradient descent. A smaller value of α will reduce the speed of computation. On the other hand, a larger value of α may skip the converging point(the situation is called as overshooting).

The screenshot from Andrew NG's course explains both issues perfectly.

In the equation,

θj:=θj−α∂θj∂J(θ0,θ1)  
the derivative part or slope will reduce and reaches zero at the minimum value. i.e,

θj:=θj−α * 0 =>
𝜃 = 𝜃𝚥 => Minimum value.






Location: United States

Comments

Post a Comment

Popular posts from this blog

Unsupervised Learning

Image
In unsupervised learning, we are provided with a dataset and nothing else. There is no outputs, parameters or anything to distinguish the given data. Our aim is to let the machine to learn or create a grouping/classification on its own. Unsupervised learning is comparatively difficult with respect to supervised learning. Consider social networking sites, based on our actions they are filling our feed with posts/media that we would prefer, suggests people we want to follow and a lot of things. These kinds of solutions are done using unsupervised learning. In the figure, social media users are grouped by machine based on some factors which are unknown to the programmer. This could be a simple example of unsupervised learning.
Read more

Automate Blog Post creation using Blogger APIs and Python

Image
This article will discuss the following topics, Blogger's REST APIs. Google's OAuth 2.0 playground. Accessing spreadsheet in Python. Creating blogger posts via Python. An experiment to automate blog post creation when you have structured data. I was searching for IFSC codes for some bank transactions and came across Reserve Bank's website, which has a list of Excel files containing Bank Codes, Contact Info and related details of each bank's branch. This time I will be looking towards Bharat co-operative bank of Mumbai. Here is how the corresponding data looks like. In the above picture, each row represents lots of details about each branch of the Bank. Let's think of creating a list of blog posts where each post gives IFSC codes of a particular branch along with address and contact details. As the first step, I have created a blog in blogger. In the Blogger dashboard, please also note the parameter called blog ID which is a kind of unique identifier ...
Read more

Setting up Python Flask server on internet via Port forwarding

Image
In this post we will cover topics; Starting a server using Python Flask. Accessing server in Local Network. Adding a Port Forwarding rule in NAT settings. Accessing server in Internet via public IP. Difference between static IP and dynamic IP. Step 1 :- As the first step, we need to setup a server on our system.A server exposes your code to a particular port.Some examples are Apache Tomcat, Spring boot (inbuilt), Python Django (inbuilt), Nginx. Here we have a simple server in Python Flask. For a GET request in root path the server returns a "Hello World!".That's it. Going to start the server on my local IP address(192.168.1.105) at default flask port(5000) using below commands, $export FLASK_APP=flaskTest.py $flask run  --host=192.168.1.105 Step 2 :- For security purpose all ports for external communication is closed by router's firewall by default. So no communication can be done to outside world. For same reason our server won't be...
Read more

Install Docker on Debian 12 (Bookworm)

Image
How to Install Docker on Debian Docker is a popular platform for developing, shipping, and running applications inside containers. This guide will show you how to install Docker on a Debian-based system. Step 1: Update the Package Index and Install Necessary Packages sudo apt-get update sudo apt-get install \ ca-certificates \ curl \ gnupg \ lsb-release Step 2: Add Docker’s Official GPG Key sudo mkdir -p /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg Step 3: Set Up the Docker Repository echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null Step 4: Update the Package Index Again sudo apt-get update Step 5: Install Docker Engine, containerd, an...
Read more

Transferring Data Between PostgreSQL Docker Containers

Transferring Data Between PostgreSQL Docker Containers Docker containers provide an efficient way to package, distribute, and run applications, including databases like PostgreSQL. Often, you may need to transfer data from one PostgreSQL container to another, either for backup, migration, or testing purposes. In this guide, we'll walk through the process of transferring data between PostgreSQL Docker containers. Prerequisites Basic understanding of Docker and PostgreSQL. PostgreSQL Docker containers already running. Step 1: Export Data from the Source Container First, we need to export the data from the source PostgreSQL Docker container. We'll use the pg_dump tool for this purpose. docker exec -it <source_container_name> bash pg_dump -U <username> -d <database_name> > /path/to/export/dump.sql exit Replace <source_container_name> , <username> , <database_name> , and /path/...
Read more