Abstarct: Because of the destructive effects of chemotherapy and the moderating its effect by the creation of the drug resistance, we seek a way to simulate the behavior of cancer cells in the face of the immune system, healthy cells and medications, in order to help the medical team find the most effective pharmaceutical compounds for cancer patients. In present thesis, we propose a mathematical model of cancer with unknown parameters. then study the chaos and optimal control of cancer model with complete unknown parameters. The linear stability analysis of the biologically feasible equilibrium states of this model will be discussed. The ranges of the system parameters for which the system has chaotic behavior will be found. The estimation of the system unknown parameters will be presented. The present thesis has been arranged in ۵ chapter. This first chapter has be summarize to introduction definition of dynamic system and the conceptions which is needed in the next chapters. In chapter ۲, we have a general look at chaos theory and chaotic systems control. In chapter ۳, we will have a brief look at the calculus of variations and optimal control problem and optimization conditions. In chapter ۴, we propose a mathematical model of cancer with unknown parameters and we use numerical methods. In chapter ۵, we conclude.