watch casio india

we are considering solution of non-linearequations. last lecture we showed that, in the newton's method we have got quadraticconvergence. now, today, we are going to look at the order of convergence in secant method,then we will consider a method which is known as regula falsi method. in the secant methodwe may not have bracketing of 0s, so this we modify the method, so that we are goingto get an interval in which our 0 is going to lie. and then, we will consider what isthe drawback of regula falsi method, after this we will consider iterative solution ofsystem of linear equations. so, we have already considered direct solution,so those are the gauss elimination method and its variants. so, today, we will considertwo methods which are iterative in nature

and those are the jacobi method and the gaussseidel method; we will give sufficient conditions under which these iterative solutions theywill converge to the exact solution of system of linear equation. so, let us look at theerror in the secant method. so, in the secant method this is the definition,x 0 and x 1, these are the initial points and then we define x n plus 1 is equal tox n minus f x n upon divided difference of f based on the earlier to approximations xn and x n minus 1; the formula is valid for n is equal to 1 and so on. suppose, that fhas a simple 0 at c, that means, f of c is equal to 0, but f dash c is not equal to 0,this f of c is going to be equal to f of x n plus divided difference based on x n x nminus 1 multiplied by c minus x n and the

error term; so, the error term is given bythe divided difference of f based on x n x n minus 1 c multiplied by c minus x n c minusx n minus 1. so, what we are doing is, we are looking ata linear approximation of f x based on interpolation, based on x n and x n minus 1, and this isthe error, and we are putting x is equal to c. now, you divide throughout by f of x nx n minus 1 and the term f x n divided by this divided difference plus c minus x n,you will take it on the other side. so, when we do that, on the left hand sidewe will have x n minus f x n divided by the divided difference of f based on x n x n minus1 minus c, so this is the term divided by the divided difference taken on the left handside; and on the right hand side you are left

with f of x n x n minus 1 c divided by thedivided difference based on x n x n minus 1; and now for c minus x n i am writing theerror e n c minus x n minus 1 is the error e n minus 1.if you look at this term, this is nothing but our x n plus 1, so you have got x n plus1 minus c is equal to if your function is two times differentiable then the divideddifference in the numerator will be f double dash d n by 2, the divided difference in thedenominator will be equal to f dash of r n, where both these d n and r n they will liein the interval bounded by x n minus 1 and x n; multiplied by e n multiplied by e n minus1; so, here on the left hand side what you have is minus of e n plus 1; so, take themodulus of the both the sides, so you will

have modulus of e n plus 1 is equal to modulusof this quotient into mod n into mod e n minus 1. so, thus modulus of e n plus 1 is equal tomodulus of f double dash d n upon two times modulus of f dash r n into mod e n into mode n minus 1. if i call the quotient to be equal to alpha n, then limit of alpha n asn tends to infinity will be equal to modulus of f double dash c divided by two times modulusof f dash c. so, this is the error in the secant method.now, recall the error in the newton's method; there we had modulus of e n plus 1 is equalto some beta n and then mod of e n square, so, this was for newton's method; and forthe secant method we have got modulus of e

n plus 1 is equal to alpha n mod n mod moden minus; so, what was earlier mod e n square, 1 mod e n is replaced by modulus of e n minus1. so, here in the case of newton's method wecould show that limit of modulus of e n plus 1 divided by mod e n square as n tends toinfinity is equal to m not equal to 0, so that is what gave us quadratic convergence;now, because instead of mod e n 1 mod e n we have got e n minus 1, we will not get ashigh order of convergence at quadratic, but what we will have will be that, it can beshown that the order of convergence in the case of newton's method will be 1.618. now, this part i am going to skip. so, youare going to have limit as n tends to infinity

modulus of e n plus 1 by mod e n raise top is equal to m which is not equal to 0 and so this p is going to be better than the linearconvergence; in the case of linear convergence we have got p is equal to 1, whereas now youget p to be 1.618, so not as good as quadratic, but better than the linear convergence. now, look at this formula for secant method.so, x n plus 1 is x n minus f x n upon f x n x n minus 1, so i substitute for the divideddifference f x n minus f x n minus 1 divided by x n minus x n minus 1 and simplify thisterm to get x n minus 1 f x n minus x n f of x n minus 1 divided by f x n minus f xn minus 1. now, nowhere in the secant method we are makingany use of the sign of f xn f xn minus 1,

so there is no restriction as such; so, thisf x n and f x n minus 1 they can be of the same sign, if they are of the same sign. so,see look at here you have got f x n minus f x n minus 1, these both f x n and f x nminus 1 they are going to converge to f of c, so you will be divide you will be subtractingtwo numbers which are of the same magnitude and then it can be prone to the round-offerrors. so, now, in order to take care of this oras a remedy what we can do is, you start with x 0 and x 1, so these two x 0 and x 1 youcan make sure that fx 0 and fx 1, they are of opposite sign. then you look at the intersectionof the secant passing through point's fx 0 and fx 1 with the x axis, whatever is thatintersection that is our point x 2; so, now,

we have got three points, fx 0, fx 1, andfx 2; earlier what we were doing was or in the secant method what we are doing is, weconsider now x 1 and x 2; now, instead of that, from x 0, x 1, x 2, you look at theif f x 1 and f x 2 are opposite of opposite signs, then choose your points to be x 1 andx 2; if they are of the same sign, then you can choose your points to be x0 and x2, sothis is known as regula falsi method. so, let me describe the method. so, our assumption is that, f from a b tor is continuous, f a into f b is less than 0, you said a 0 is equal to a b 0 is equalto b, and then you look at w which is a n times f b n minus b n times f a n upon f bn minus f a n; so, how do we get this formula?

this is nothing but the intersection of thesecant passing through the point's f a n and f b n with the x axis, so this is our w; iff a n f w is less than 0, that means, if they are of opposite signs, then you choose youra n plus 1 to be equal to a n and b n plus 1 to be equal to w; if they are of the samesign, then choose n plus 1 to be w, and b n plus 1 to be equal to b n.so, it is similar to the secant method; only at every step we make sure that our f a nand f b n they are going to be of opposite sign, so that means, we can say that our 0is going to lie in the interval a n to b n, this is not the case with secant method wehave got x n x n plus 1. so, when there is convergence both f x n and f x n plus 1, theywill converge to f of c or they will converge

to 0, but the interval x n to x n plus 1,it need not contain c, so we do not have bracketing of 0 in the secant method. now, here in theregula falsi method because we make sure that f a n and f b n they are of opposite signs,our interval a n to b n is going to contain our point c, the point c is where f of c isequal to 0; but in this case also there is a disadvantage, so the disadvantage i wantto explain by a graph. suppose, your function is what is known asconcave up, that means, when you take any two points on the curve and join them by straightline, this secant it lies above the graph of your function this is satisfied; for example,if f dash x is bigger than 0, f double dash x is bigger than or equal to 0, so let melook at the regula falsi method; so, here

you have i start with a 0 and point b 0, ilook at the straight line then it intersects the x axis at a 1, so that is my new pointa 1, so we have a 0 a 1 and this is b is our b 0.if i look at a 0 and a 1, they are of the same sign, both of them they are negative;so, what i am going to do is, i am going to choose my points to be a 1 and b 1, b 1 issame as b 0. now, you look at the secant joining f of a 1 and f of b 1, now once again becauseit lies above the x axis, your a 2 point is such that f a 2 and f a 0 they are f a 2 andf 1 rather we should compare f a 1 and f a 2, so they will be of the same sign; so, thatmeans, you should choose the point to be a 2 and b, so you are going to have a 2 b 2,so which is identical to b then a 3.

so, here the point of intersection w it willlie always to the left of c, that means, your right hand side is always going to be equalto b n plus 1 is equal to b, so that means, the length of the interval it may not tendto 0. so, in secant method we dint have bracketing of 0s.in case of regula falsi method our a n to b n the interval a n to b n it is going tocontain our point c our point our 0, but it can happen like i showed you graphically thatone of the points of your interval that keeps changing, but the b n it remained the original,whatever was our x 0 and x 1; so, in that if i call x 1 to be equal to b it remainsthe same; so, the length of the interval it may not shortened, so it may not tend to...,so you get bracketing of 0, but that length

may not be reducing as fast as we wish.in case of bisection method at least we know that at each stage our interval gets reducedby half, this may not happen in case of regula falsi method, so there are advantages anddisadvantages. so, let me compare now our newton's method and secant method; we havealready considered there plus points and minus points, so i just want to summaries aboutthe two methods. so, in the case of newton's method we havegot x n plus 1 is equal to x n minus f x n upon f dash x n. so, at each stage you needto evaluate the function and the derivative; so, that means, we have got two function evaluationsper step, you have got quadratic convergence that is the biggest advantage of newton'smethod.

the derivatives are involved, so if it iscomplicated to calculate the derivative, then newton's method will not be recommended. inthe case of secant method you have got x n plus 1 is equal to x n minus f x n dividedby this divided difference; so, in order to calculate the divided difference you needto evaluate f x n and f x n minus 1, but you would have in the earlier step you would haveevaluated f x n minus 1 and f x n. so, let me tell you what it is. so, you havegot x n plus 1 is equal to x n minus f x n divided by f x n minus f x n minus 1 dividedby x n minus x n minus 1, this is for x n plus 1. so, you have x n is equal to x n minus1 minus f x n minus 1 divided by f of x n minus 1 minus f of x n minus 2 divided byx n minus 1 minus x n minus 2. so, that means,

this f xn minus 1 is common to evaluationof x n as well as x n plus 1 and that is why in the case of secant method you are goingto essentially have one function evaluation per step except for the first step. in the first step you will need to evaluatefx 0 as well as fx 1; so, when you calculate or when you consider the number of computationsfor secant method, they are half, here you have two function, here you have got one functionevaluation per step. the convergence is not as fast as quadratic convergence; you do notneed to calculate the derivatives; so, there are some advantages with the newton's methodand some advantages with the secant method. so, now we say that the newton's method wehave got quadratic convergence; this is under

the assumption that f as a simple 0, thatmeans, f of c is equal to 0, but f dash c is not equal to 0. now, suppose, the functionhas two 0s or it as a double 0 rather, that means, you have got f of c is equal to 0 fdash c is equal to 0, but f double dash c not equal to 0in this case, we will see that the quadratic convergence in the newton's method gets reducedto the linear convergence, but you can modify your newton's method so as to retain quadraticconvergence also in the case of a double 0, so this is what now i am going to explain.we got quadratic convergence in newton's method, because of the fact that when we look at gx to be equal to x minus f x upon f dash x. so, fix point iterations for this particularfunction is nothing but the newton's method

and this particular function it has the propertythat g dash c is equal to 0. so, we are going to relate the newton's method to fix pointiteration and then that gives us a clue as to how to modify newton's method in case ofmultiple 0s so as to retain the quadratic convergence. i will take the case when itis a double 0; if it is a triple 0 or 0 of multiplicity m then the modification is similar. so, let us look at the order of convergencein newton's method. so, look at g x is equal to x minus f x upon f dash x; then newton'smethod is nothing, but fix point iteration for this particular function. if f of c isequal to 0, f dash c is not equal to 0, then we had seen this before g dash x is nothingbut f x f double dash x by f dash x square,

we have got f of c is equal to 0, so thatwill give you g dash c to be equal to 0. then you consider x n plus 1 minus c, thiswill be x n plus 1 by definition is g x n, c being a fixed point of g it will be equalto g of c write down the taylor's series expansion, so that will be equal to x n minus c intog dash c plus x n minus c square by 2 g double dash d n x n plus 1 minus c is e n plus 1then x n minus c square will be e n square g double dash d n by 2, so that is what givesyou quadratic convergence in case of newton's method. now, suppose, it has a double 0, c is suchthat f of c is equal to f dash c is equal to 0 and f double dash c not equal to 0. thenyou have g x is as before x minus f x upon

f dash x. so, we are looking at the case whenf of c is equal to f dash c is equal to 0. and you have got f double dash c to be notequal to 0; g x is x minus f x upon f dash x, so, we have got g of c is equal to c; theng dash x when we look at it will be f dash x square into f x f double dash x. so, wheni look at limit of g dash x as x tends to c, this will be equal to limit of f doubledash x as x tends to c and limit of f x upon f dash x square as x tends to c; the firstpart is going to be equal to f double dash c; and now, here f of c is 0 f dash c is 0,so it is going to be of the 0 by 0 form. so, we have to apply l'hopital's rule andthat will give you limit x tends to 0, so take the derivative of the numerator thatis f dash x and derivative of the denominator

will be 2 f dash x f double dash x, so thatmeans, this limit is going to be equal to 1 by 2. so, in case of simple 0, we had g dash c isequal to 0; now, we have got limit of g dash x as x tends to c to be equal to half, thisis what will make the convergence to be linear convergence if c is a double 0. so, what wewant to do is, we want to modify this function g, the reason we got quadratic convergencein case of simple 0 was g dash of c is equal to 0; so, under these conditions i want tomodify my function g such that g dash c is going to be equal to 0. we have got limit of g dash x as x tends toc is equal to half and hence if i define look

at function g x which is equal to x minus2 f x upon f dash x, then what will happen will be when i look at g dash x, here it is1 then minus 2 and limit of this we have seen that it is half, so there will be minus 2into half, so that will be 1 and then you are going to get limit of g dash x as x tendsto c is equal to 0. so, thus if you make this simple modificationthat, if you consider your x n plus 1 to be equal to x n minus 2 f xn upon f dash xn,then you are going to have quadratic convergence; if it is a triple 0 then instead of 2 hereyou put 3; if it is a 0 of multiplicity m, then you put here m. so, now, this was about the solution of non-linearequation one single equation, one can consider

system of non-linear equations, but let usfirst look at system of linear equations. so, let us go back to our system of linearequations and consider iterative methods. so, our setting is going to be n equationsin n unknowns, the coefficient matrix a is invertible, so we have got a x is equal tob, a is equal to a i j n by n invertible matrix, this is our additional assumption that a ii the diagonal entries of a they are going to be not equal to 0 for i is equal to 1 toup to n; this a x is equal to b it is nothing but summation a i j x j, j going from 1 ton equal to b i, i goes from 1 to up to n; these are the n equations, the right handside b is given to us the coefficient matrix a is given to us x the vector x 1 x 2 x nthat is unknown.

since a i i is not equal to 0, we can writethis as x i is equal to b i minus summation j goes from 1 to n a i j x j j not equal toi divided by a i i, i going from 1 to up to n. so, it is the same system of linear equations,i am writing in a different manner. so, now, in the case of iterative methods what we aregoing to do is, we will start with some initial approximation, so x 0 is going to be our initialvector, you can choose that vector to be equal to 0 0 0. so, start with a 0 vector. so, wehave written the equations which exact solutions satisfy, that we had x i is equal to b i minussummation over j a i j x j except j not equal to i divided by a i i.so, now, on the right hand side you put values for x j to be from the earlier one, so likeconsider the starting iteration x 1 0 x 2

0 x n 0. so, put it on the right hand sideand whatever new value you get that is going to be your x i 1, so this is known as thejacobi method; and for this method we will give a sufficient condition under which theiterates will converge to the exact solution; these iterative methods become important whensolving the system of linear equations by direct methods become expensive, because wehave seen that the direct solution is of the order of n cube by 3, so if n is very largethen it can be very expensive. also in these direct methods we do not reallymake use of the fact that if your coefficient matrix a happens to be spars; if it has alot of 0 still your gauss elimination method is going to cost n cube by 3 except if your0s are structured, that means, if your matrix

is a coefficient matrix is a tri-diagonalmatrix, then instead of the operations of the size n cube they will reduce to the operationsof the size n; but the methods which i am going to describe now they are useful whenyou have a lot of 0s, but they are they do not have a pattern like tri-diagonal or somethinglike that, but still there we have a lot of 0s; in that case these methods they can beuseful, and of course, there will be sufficient conditions we have to have if the some conditionsare satisfied then we are going to get convergence. so, essentially, if your matrix is diagonallydominant, then the jacobi method as well as the gauss seidel method they will converge. so, here is the description of the methodthat a x is equal to b, this is the equation

satisfied by the exact solution start withthe initial approximation and then x i k is equal to b i minus summation j goes from 1to n j not equal to i a i j x j k minus 1. so, these you have obtained from the earlierone divided by a i i i goes from 1 to up to n. so, you calculate x i 1 or rather x 1 k,x 2 k, x n k, once you have calculated all of them then you go to the go to calculationof x1 k plus 1, because now on the right hand side you will know all x j k. so, this is the jacobi method. and now, letus look at the error. so, when i consider the x i minus x i k, when i subtract these2 the b i's will get cancelled. so, you are left with summation j goes from 1 to n j notequal to i a i j by a i i x j; and here similar

thing with instead of x j x j k minus 1 andsummation over j a i j by i i j not equal to i. so, i subtract the two equations and i getx i minus x i minus k to be summation j goes from 1 to n a i j upon a i i x j minus x jminus x j k minus 1. here there should be a minus afterwards, becausewe are going to take modulus it will not matter, but here there is one extra minus sign. thenwe look at the maximum norm. so, norm of x minus x k its infinity norm is maximum ofmodulus of x i minus x i k 1 less than or equal to i less than or equal to n; x is theexact solution it is a n by n vector; x k is the kth iterate, it is also a n by 1 vector;now, from here you will get modulus of x i

minus x i k to be less than or equal to...let me dominate x j minus x j k minus 1 its modulus by norm of x minus x k minus 1 itsinfinity norm. so, if i dominate by this norm it will come out of the summation sign, soyou are left with summation j goes from 1 to n j not equal to i modulus of a i j bya i i. now, norm of x minus x k infinity will be less than or equal to maximum of this summation,the maximum is over i, so summation is over j goes from 1 to n j not equal to i; so, thisnumber depends on i and then you are taking its maxima into norm of x minus x k minus1 infinity, let me call this number as mu. so, we have got the error in the kth iterateto be less than or equal to mu times error in the k minus first iterate. so, then replacingk by k minus 1, this will be less than or

equal to mu times the error in k minus seconditerate and so on. so, you will get norm of x minus x k infinityto be less than or equal to mu raise to k norm of x minus x 0 its infinity norm.if your mu is less than 1, then mu raise to k will tend to 0 as k tends to infinity andthe vector x k will tend to x as k tends to infinity, x is the exact solution. so, thismu less than 1 it means summation j goes from 1 to n modulus of a i j, j not equal to ishould be less than modulus of a i i, these are the diagonal entries, these are the halfdiagonal entries in the same row. so, mu less than 1, that means, strictly domin the strictlydiagonally row dominant; so, if this is the case then your jacobi method is going to converge.so, in case of jacobi method what we do is,

we start with an initial vector. so, you havegot x 1 0, x 2 0, x n 0, 0 is the super script. then using the formula you calculate the firstiterates; so, you calculate x 1 1, x 2 1, x n 1; so, you are going to calculate do thecalculations in the order. so, we have x 1 0, x 2 0, and x n 0, thisis our initial approximation. then you calculate x 1 1, x 2 1 and x n 1, so, this is your nextstep in the jacobi method. so, when actually you calculate x 2 1, you have values availableof x 1 0, and x1 1, but we do not use this value, we calculate all of these x 1 1, x2 1, x n 1. so, in the gauss seidel method what one does is, when i want to calculatex 2 1, then my x 2 1 is going to be b i minus summation j going from 1 to n j not equalto 2 a i j x j 0 divided by a i i, so i have

got a term b i minus a, here i should be equalto 2, and here it is a 2 2; so, it is going to be minus a 2 1 x 1 0, but i have i am goingto go in order, so for this x 1 0 actually i have calculated x 1 1, so why not use thatmore recent value; so, if you do that then you get the gauss seidel method like whenyou are calculating say x n 1 the last one, then for the x n 1 it will be in the caseof jacobi method what we do is to use the values x 1 0, x 2 0, x n minus 1 0.but now, actually hopefully better approximations are available, so why not use those approximations;so, at any stage whatever are the recent values of the approximations, whatever are availablelike when consider x 2 1, it will also need x 3 0 and then x 3 1 is not available. so,whatever is whatever recent values are available,

use those values and then one hopes that thatshould give you better approximation no one can construct pathological examples whereyou would not have any improvement. in general, you will have improvement andwhat i am going to do is, i am going to show that, we had a sufficient condition for convergenceof jacobi method, so that was mu should be less than 1. now, in case of gauss seidelmethod we will have another sufficient condition satisfied for the condition for the convergenceof the iterates. so, we will show..., so in that case suppose eta is less than 1. so,what we will show is mu less than 1 implies eta less than or equal to mu less than 1.we are not saying that whenever jacobi method converges guass seidel method has to converge,that is false; what i am saying is, we are

going to obtain two sets of sufficient condition,one for the jacobi method, another for the gauss seidel method; if you compare thesesufficient conditions, then if the sufficient condition in the jacobi method is satisfied,it will imply that the sufficient condition in the gauss seidel method also will be satisfied.so, let me first describe what is gauss seidel method obtain a sufficient condition for theconvergence and then compare the sufficient conditions in the jacobi method and in thegauss seidel method. so, as before our exact solutions satisfiesx i is equal to b i minus summation over j j not equal to i a i j x j divided by a ii, i goes from 1 to up to n. so, let me split this summation as j going from 1 to i minus1 and j is equal to i plus 1 to n, our summation

is from 1 to n except for the term j not equalto i, so, except for the term j is equal to i; so, this summation i am splitting whenwe define the iterations; for these sum we will use the recent value available; and forthis we will use the earlier value from the iterate.now, here the convention which we are following is, if i is equal to 1, then this term willnot be there, it is from it will become from j is equal to 1 to 0, so our convention isthis term will not be there; if i is equal to n, then this term will not be there; likewhen i is equal to 1 then you are going to have x1 is equal to b 1 minus summation jgoes from 2 to n; if i is equal to n, it will be x n is equal to b n minus summation j goesfrom 1 to n minus 1.

so, here is the definition, these are thekth iterate. so, you have x i k is equal to b i minus summation j goes from 1 to a i iminus 1 a i j x j k. so, we have already calculated x1 k, x2 k, x i minus 1 k, so use those recentvalues here and then minus summation j is equal to i plus 1 to n a i j x j k minus 1.so, at this stage, when you look at x i k x1 k up to x i minus 1 k are available, souse those values, and here you have no choice, but you have to use the values from the earlieriterate. so, if it was a jacobi iterate, then here also it would have been x j k minus 1.so, here this is the gauss seidel iterate, look at x i minus x i minus k subtract thetwo, you are going to have minus summation j goes from 1 to i minus 1 a i j by a i ix j minus x j k minus summation j is equal

to i plus 1 to n a i j by a i i x j minusx j k minus 1, so this is e i k, this will be e j k, and this will be e j k minus 1. so, e i k is equal to minus this summationminus this summation. so, let me define alpha i to be summation j goes from 1 to i minus1 modulus of a i j by a i i, beta i to be summation j going from i plus 1 to n modulusof a i j by a i i. define alpha 1 to be 0 and beta n to be 0, so you will get modulusof e i k to be less than or equal to alpha i times norm e k infinity plus beta i timesnorm of e k minus 1 infinity. so, now, as we did in the case of jacobi method,we will be trying to relate norm e k infinity with norm e k minus 1 infinity; here it becomesslightly more complicated, but not much more.

so, we will continue next time and we willobtain a sufficient condition for convergence of gauss seidel method. so, thank you.