Abstarct: Recent advancements in natural language processing have resulted in the creation of language models that can mimic human-written text very well, when used as text generation models. These models can be used for malicious purposes such as fake news generation, and measures must be taken to counter their threats. A typical text generation model works by taking an initial token or sequence as input, and generating and appending tokens to the sequence until a special ‘end’ token is generated. In each step of token generation, the model computes a probability distribution over all tokens in its vocabulary, given the sequence generated so far, as input. This distribution represents the probability of each token appearing in the sequence next, and a final token is chosen from this distribution using a sampling method and appended to the sequence. One method for detecting AI generated text is training a binary classifier on human generated and AI generated text. As the sampling method used for text generation can have a great impact on the model’s performance on detecting machine generated text, we evaluate all finetuned classifier models on two sets of machine-generated text that were produced using different sampling methods. In previous work, the BERT model wa finetuned for Human-AI text classification, but less focus has been put on using more improved architectures for classification of machine generated text. In this work we finetune an Electra language model and a BERT language model for Human-AI text classification, and show that the Electra model improves the accuracy of BERT on detecting machine text generated using random sampling and top-k sampling by 4 percent and 1 percent respectively on the gpt2-output dataset.