Preface |
|
xiii | |
Part I Mathematical Review |
|
|
Methods of Proof and Some Notation |
|
|
1 | (4) |
|
|
1 | (2) |
|
|
3 | (2) |
|
|
4 | (1) |
|
Vector Spaces and Matrices |
|
|
5 | (16) |
|
|
5 | (5) |
|
|
10 | (4) |
|
|
14 | (2) |
|
|
16 | (5) |
|
|
19 | (2) |
|
|
21 | (18) |
|
|
21 | (1) |
|
Eigenvalues and Eigenvectors |
|
|
22 | (3) |
|
|
25 | (1) |
|
|
26 | (5) |
|
|
31 | (8) |
|
|
35 | (4) |
|
|
39 | (10) |
|
|
39 | (1) |
|
Hyperplanes and Linear Varieties |
|
|
39 | (3) |
|
|
42 | (2) |
|
|
44 | (1) |
|
|
45 | (4) |
|
|
47 | (2) |
|
|
49 | (24) |
|
|
49 | (6) |
|
|
55 | (2) |
|
|
57 | (2) |
|
|
59 | (1) |
|
|
60 | (4) |
|
|
64 | (9) |
|
|
68 | (5) |
Part II Unconstrained Optimization |
|
|
Basics of Set-Constrained and Unconstrained Optimization |
|
|
73 | (18) |
|
|
73 | (2) |
|
Conditions for Local Minimizers |
|
|
75 | (16) |
|
|
83 | (8) |
|
One-Dimensional Search Methods |
|
|
91 | (22) |
|
|
91 | (4) |
|
|
95 | (8) |
|
|
103 | (3) |
|
|
106 | (2) |
|
Remarks on Line Search Methods |
|
|
108 | (5) |
|
|
109 | (4) |
|
|
113 | (26) |
|
|
113 | (2) |
|
The Method of Steepest Descent |
|
|
115 | (7) |
|
Analysis of Gradient Methods |
|
|
122 | (17) |
|
|
122 | (7) |
|
|
129 | (5) |
|
|
134 | (5) |
|
|
139 | (12) |
|
|
139 | (3) |
|
Analysis of Newton's Method |
|
|
142 | (3) |
|
Levenberg-Marquardt Modification |
|
|
145 | (1) |
|
Newton's Method for Nonlinear Least-Squares |
|
|
146 | (5) |
|
|
149 | (2) |
|
Conjugate Direction Methods |
|
|
151 | (16) |
|
|
151 | (2) |
|
The Conjugate Direction Algorithm |
|
|
153 | (5) |
|
The Conjugate Gradient Algorithm |
|
|
158 | (3) |
|
The Conjugate Gradient Algorithm for Non-Quadratic Problems |
|
|
161 | (6) |
|
|
164 | (3) |
|
|
167 | (20) |
|
|
167 | (1) |
|
Approximating the Inverse Hessian |
|
|
168 | (3) |
|
The Rank One Correction Formula |
|
|
171 | (5) |
|
|
176 | (4) |
|
|
180 | (7) |
|
|
184 | (3) |
|
|
187 | (32) |
|
|
187 | (9) |
|
Recursive Least-Squares Algorithm |
|
|
196 | (3) |
|
Solution to Ax = b Minimizing ∥x∥ |
|
|
199 | (2) |
|
|
201 | (3) |
|
Solving Ax = b in General |
|
|
204 | (15) |
|
|
212 | (7) |
|
Unconstrained Optimization and Neural Networks |
|
|
219 | (18) |
|
|
219 | (2) |
|
|
221 | (3) |
|
Backpropagation Algorithm |
|
|
224 | (13) |
|
|
234 | (3) |
|
|
237 | (18) |
|
|
237 | (6) |
|
Chromosomes and Representation Schemes |
|
|
238 | (1) |
|
|
238 | (5) |
|
Analysis of Genetic Algorithms |
|
|
243 | (5) |
|
Real-Number Genetic Algorithms |
|
|
248 | (7) |
|
|
250 | (5) |
Part III Linear Programming |
|
|
Introduction to Linear Programming |
|
|
255 | (32) |
|
A Brief History of Linear Programming |
|
|
255 | (2) |
|
Simple Examples of Linear Programs |
|
|
257 | (6) |
|
Two-Dimensional Linear Programs |
|
|
263 | (1) |
|
Convex Polyhedra and Linear Programming |
|
|
264 | (3) |
|
Standard Form Linear Programs |
|
|
267 | (5) |
|
|
272 | (4) |
|
Properties of Basic Solutions |
|
|
276 | (3) |
|
A Geometric View of Linear Programs |
|
|
279 | (8) |
|
|
282 | (5) |
|
|
287 | (34) |
|
Solving Linear Equations Using Row Operations |
|
|
287 | (7) |
|
The Canonical Augmented Matrix |
|
|
294 | (1) |
|
Updating the Augmented Matrix |
|
|
295 | (2) |
|
|
297 | (6) |
|
Matrix Form of the Simplex Method |
|
|
303 | (4) |
|
The Two-Phase Simplex Method |
|
|
307 | (3) |
|
The Revised Simplex Method |
|
|
310 | (11) |
|
|
315 | (6) |
|
|
321 | (18) |
|
|
321 | (7) |
|
Properties of Dual Problems |
|
|
328 | (11) |
|
|
333 | (6) |
|
|
339 | (26) |
|
|
339 | (1) |
|
|
340 | (3) |
|
|
343 | (5) |
|
|
343 | (4) |
|
|
347 | (1) |
|
|
348 | (17) |
|
|
348 | (1) |
|
Karmarkar's Canonical Form |
|
|
349 | (2) |
|
Karmarkar's Restricted Problem |
|
|
351 | (1) |
|
From General Form to Karmarkar's Canonical Form |
|
|
352 | (4) |
|
|
356 | (4) |
|
|
360 | (5) |
Part IV Nonlinear Constrained Optimization |
|
|
Problems with Equality Constraints |
|
|
365 | (32) |
|
|
365 | (1) |
|
|
366 | (2) |
|
Tangent and Normal Spaces |
|
|
368 | (6) |
|
|
374 | (10) |
|
|
384 | (3) |
|
Minimizing Quadratics Subject to Linear Constraints |
|
|
387 | (10) |
|
|
391 | (6) |
|
Problems with Inequality Constraints |
|
|
397 | (20) |
|
Karush-Kuhn-Tucker Condition |
|
|
397 | (9) |
|
|
406 | (11) |
|
|
410 | (7) |
|
Convex Optimization Problems |
|
|
417 | (22) |
|
|
417 | (2) |
|
|
419 | (8) |
|
Convex Optimization Problems |
|
|
427 | (12) |
|
|
433 | (6) |
|
Algorithms for Constrained Optimization |
|
|
439 | (16) |
|
|
439 | (1) |
|
|
439 | (2) |
|
Projected Gradient Methods |
|
|
441 | (4) |
|
|
445 | (10) |
|
|
451 | (4) |
References |
|
455 | (7) |
Index |
|
462 | |