Wednesday, August 17, 2022

Autoscaling in Kubernetes

Kubernetes pod autoscaler using custom metrics – Sysdig 

Autoscaling in Kubernetes:

ReplicaSet works with a set number of pods.

Horizontal Pod Autoscaler(HPA) enables scaling up and down as needed

Can configure based on desired state of CPU, memory etc.

 

Horizontal Pod Autoscaler, or HPA, enables the application to increase the number of  pods based           on traffic.

 

The master node will periodically check pod metrics and scale to meet the desired state by updating   the replicas field of the scaled resource, such as ReplicaSets or deployment. 

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
labels:
name: nginx-deployment
annotations:
kubernetes.io/change-cause: "custom message"
spec:
replicas: 5
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80

$ kubectl apply -f deployment.yaml

$ kubectl get all



$ kubectl get hpa  

$ kubectl autoscale deploy nginx-deployment  --min=2  --max=5 --cpu-percent=10

"cpu-percent" is the trigger to create new pods.  This tells the system, “If the CPU usage hits 10% across the cluster, create a new  pod.”


 


 

A Horizontal Pod Autoscaler is created behind the scenes to manage the  autoscaling feature of the deployment.

 

 

 

 

 

 

 

 

 

 

Monday, July 18, 2022

Docker Compose 

  • Docker Compose is used for running multiple containers as a single service. 
  • Each of the containers here run in isolation but can interact with each other when required.
  • Kubernetes and Docker Compose are both container orchestration frameworks. 
  • Kubernetes runs containers over a number of computers, virtual or real.  
  • Docker Compose runs containers on a single host machine.
     

     
 File : docker-compose.yml
version: '3'
services:
webapp1:
image: nginx
ports:
- "8000:80"
webapp2:
image: nginx
ports:
- "8002:80" 
 
 

NOTE:Default file name is docker-compose,you can define your docker-compose file using below syntax

docker-compose -f docker-compose2.yml up -d
(-d for background ) 
Commands:

docker-compose create ( depreciated ,create only container and does not create network)
docker-compose up --no-start  ( create only container with default network)
docker-compose rm ( to remove container )
docker-compose start ( To start container )
docker-compose stop ( To stop container )

docker-compose rm ( To delete container ) 
docker-compose images ( To view images )
docker container ls 
docker-compose ps ( To check state of container )
docker-compose pause ( To pause any container )
docker-compose unpause ( To unpause any container ) 
 

docker-compose up -d  
docker-compose down

 

docker-compose kill ( To kill containers )
docker-compose start
docker-compose port webapp1 80 ( To check public accessible port )
docker-compose logs -f ( To check online logs )

docker-compose ps
docker containers ls -a

docker-compose exec webapp1 ls ( To run ls command inside running container )
docker-compose run webapp1 ls ( To run ls command inside a new  container and container went down )


docker-compose restart ( To restart container )
docker-compose pull ( Pull image from registry )
docker-compose push ( push  image to the  registry )
docker-compose --version 

 File :docker-compose.yml
version: '3'
services:
webapp1:
image: nginx
webapp2:
image: nginx
 
docker-compose scale webapp1=4 webapp2=4 ( To scale )
docker-compose top ( To view running process )
  
 To be continue....



	
	
	
	



 
 

































            

          

          

        

Friday, July 15, 2022


          

        











Kubernetes Namespaces











Namespaces are a way to organize clusters into virtual sub-clusters — they can be helpful when different teams or projects share a Kubernetes cluster. 

 

Namespaces allow to split-up resources into different groups. Resource names should be unique in a namespace. We can use namespaces to create multiple environments like dev, staging and production etc.

Kubernetes comes with three namespaces out-of-the-box. They are:

 1. default: this is the namespace that is referenced by default   for every    Kubernetes command, and where every Kubernetes resource is located by default. 
 Until new namespaces are created, the entire cluster resides in ‘default’.
 2.  kube-system: Used for Kubernetes components 
 3.  kube-public: Used for public resources. 



To create namespace

> kubectl create ns development
> kubectl create ns production

To view namespaces

> kubectl get namespaces
> kubectl get ns

Define namespace in a yaml file:
metadata:
   name  : pod4
   namespace: development


To view namespace along with associated pods

> kubectl get pods --all-namespaces

To change default namespace 

>kubectl config set-context --current --namespace=development

To delete a namespace

> kubectl delete namespace development

NOTE:Everything in the namespace including all services, running pods, and artifacts will be deleted

































            

          

          

        

Wednesday, January 20, 2021


          

        











Machine learning











 


Machine learning (ML)
is a type of artificial intelligence (AI)
that allows software applications to become more
accurate at predicting outcomes without being explicitly
programmed to do so. 
 





Machine learning algorithms
use historical data as input to predict
new output values.
 
 






Applications
of Machine Learning:





•Dynamic
Pricing



•Product
recommendations



•Traffic
prediction



•Self-driving
cars



•Email
Spam and Malware Filtering
 
 
 



Regression models 



•What
is regression?? 







•Regression
can be said to be a technique
to find out the best relationship between
the input variables known as predictors
and the output variable also known as response/target
variable.







•Best
relationship is signified by minimal
difference between the predicted
and the actual values.







•Regression
analysis is an important tool for modelling
and analyzing data. Here, we fit a curve / line to the data
points, in such a manner that the differences between the distances of data
points from the curve or line is minimized.







•It
indicates the strength of impact of
multiple independent variables on a dependent variable.
 
 



Linear regression 







•It is
a supervised learning algorithm mostly
used in predictive analysis which
typically means trying to fit the best straight line between
the input and output variables in order to model our data.







•this
best fitting straight line is also known as regression line that minimizes
the sum of the squared errors of prediction.







•Characteristic
of linear regression is the output variable should be
continuous.







•Linear
Regression establishes a relationship between dependent
variable (Y) and one or more independent variables (X) using
a best fit straight line (also known as regression
line).
  
 
 
 
 
 


Best fit - line or curve:





1. The maximum number of points covered.



2. Minimize the distance between other
points (error -SSE)
 
 
 
 
 
 
 



The
below-given equation is used to denote the linear regression model:





y=mx+c+e





where m is the slope
of the line, c is an intercept,
and e represents the error in the model





Linear regression where Y is the output
variable and X is the input variable/variables.
 
 
 



 Find Slope & Intercept:
 
 
 
 Linear Regression Python code:






 

  
  
#Importing
  Needed packages

  
import pandas
  as pd

  
import numpy
  as np

  
import
  matplotlib.pyplot as plt

  
from sklearn
  import linear_model

  
 

  
 

  
#Reading the
  data in

  
path='C:/TrainingDocs/MachineLearningwithPython/homeprices1.csv'

  
df =
  pd.read_csv(path)

  
df

  
 

  
# summarize
  the data

  
df.describe()

  
 

  
#plot graph
  for datapoints

  
%matplotlib
  inline

  
plt.xlabel('area(sqr
  ft)')

  
plt.ylabel('price(US$)')

  
plt.scatter(df.Area,df.Price,color='red',marker='+')

  
 

  
 

  
#Using
  sklearn package to model data

  
#fitting
  training data and then generating predictions on test data 

  
reg=linear_model.LinearRegression()

  
reg.fit(df[['Area']],df.Price)

  
 

  
 

  
#Predict
  price for area (3300 sq feet)

  
reg.predict([[3300]])

  
 

  
#Plot graph
  for prediction

  
#Draw the
  line on the scatter plot

  
%matplotlib
  inline

  
plt.xlabel('Area',fontsize=20)

  
plt.ylabel('Price',fontsize=20)

  
plt.scatter(df.Area,df.Price,color='red',marker='+')

  
plt.plot(df.Area,reg.predict(df[['Area']]),color='blue')

  
 

  
 

  
print
  ('Coefficients: ', reg.coef_)

  
print
  ('Intercept: ',reg.intercept_)

  
 

  
#Check Value
  of Coefficients

  
reg.coef_

  
 

  
#Check Value
  of intercept

  
reg.intercept_

  
 

  
#Validate
  linear equation

  
#y=mx+b

  
#134.07534247*3300+176232.87671232875

  
134.07534247*3300+176232.87671232875

  
 

  
#Predict
  price based on given area

  
path1='C:/TrainingDocs/MachineLearningwithPython/areas.csv'

  
d=
  pd.read_csv(path1)

  
d.head(3)

  
 

  
p=reg.predict(d)

  
d['prices'] =
  p

  
path2='C:/TrainingDocs/MachineLearningwithPython/prediction.csv'

  
d.to_csv(path2,index=False) 

  
 

  		
 





 

Multiple
linear regression:
 


Predict for 3000
sq ft, 3 bedrooms, 40-year-old



 
 
 Multiple
linear regression Python Code:






 

  
  
#Importing
  Needed packages

  
import pandas
  as pd

  
import numpy
  as np

  
import
  matplotlib.pyplot as plt

  
from sklearn
  import linear_model

  
 

  
 

  
 

  
#Reading the
  data in

  
path='C:/TrainingDocs/MachineLearningwithPython/homeprices1.csv'

  
df =
  pd.read_csv(path)

  
df

  
 

  
#Cleaning of
  data

  
import math

  
median_bedrooms=math.floor(df.Bedrooms.median())

  
median_bedrooms

  
 

  
#Assign some
  value to NaN 

  
 

  
df.Bedrooms=df.Bedrooms.fillna(median_bedrooms)

  
df

  
 

  
 

  
#To Train
  model

  
reg=linear_model.LinearRegression()

  
reg.fit(df[['Area','Bedrooms','Age']],df.Price)

  
 

  
 

  
 

  
print
  ('Coefficients: ', reg.coef_)

  
print
  ('Intercept: ',reg.intercept_)

  
 

  
 

  
 

  
#Predict for
  3000 sq ft, 3 bedrooms, 40 yeal old

  
reg.predict([[3000,3,40]])

  
 

  
 

  
#Validate
  multiple equation

  
#y=m1x1+m2x2+m3x3+y

  
#3000*137.25+3*-26025+40*-6825+383724.9999999998

  
3000*137.25+3*-26025+40*-6825+383724.9999999998

  
 

  
 

  		
 








































        

      












Subscribe to:
Posts (Atom)