Abstarct: A categorical variable is a random variable that groups observations‎. ‎For example‎, ‎classifying people with different characteristics according to their blood type results in a categorical variable‎. ‎Nowadays‎, ‎the widespread of various sciences has made analyzing categorical data very essential‎. ‎Hence‎, ‎different scientific disciplines pursue more accurate methods for analyzing such data‎. ‎In classical statistics‎, ‎several methods have been proposed to analyze categorical data‎. ‎These existing methods and relevant software packages are not efficient enough for analyzing data with complex spatial and spatio-temporal structures‎. ‎Bayesian analysis and inference of categorical data require sampling methods such as the MCMC algorithms‎, ‎while they are not also efficient for categorical data with complex structures‎. ‎Our proposed approach‎, ‎in this thesis‎, ‎is to use an approximate Bayesian method‎, ‎called integrated nested Laplace approximation (INLA)‎, ‎which is not accompanied by the significant problems seen in the MCMC algorithms for implementing the Bayesian analysis of spatial models‎. ‎To deal with the inherent identifiability problem of the multinomial models‎, ‎we consider two different types of identifiability constraints and compare their performances‎. ‎We use the Poisson-multinomial transformation to develop the model in the class of‎ latent Gaussian models applicable for INLA‎. ‎We also utilize the individualized logit model as another type of modeling depending on the response variable type‎. ‎Finally‎, ‎we assess and compare the proposed models through both simulated and real data examples‎. ‎