LSTM outperformed Random Forest and Naive Bayes in tweet sentiment analysis with a ROC-AUC of 0.92

A new study published on arXiv in July 2026 tested how much the LSTM neural network outperforms five classical machine learning algorithms in the task of analyzing the sentiment of Twitter posts — and the recurrent network won on all key metrics, achieving 80% accuracy on the test set and ROC-AUC of 0.92.

What and how was compared

The authors took a dataset of tweets from Kaggle and performed standard NLP preprocessing: tokenization (breaking into words), lemmatization (reducing to base form), and stop-word removal. After this, six algorithms were trained to classify tweets into three categories — positive, negative, and neutral.

Tested models:

• Logistic regression
• Random Forest
• Naive Bayes classifier
• Gradient Boosting
• LSTM (Long Short-Term Memory) — a recurrent neural network that accounts for word order

How well did LSTM perform?

LSTM proved to be the best of all tested approaches:

• Accuracy on training set — 90.98%
• Accuracy on test set — 80.00%
• Micro-average ROC-AUC — 0.92

An ROC-AUC value of 0.92 means that the model correctly separates positive and negative content in approximately 92% of cases — this is a high result for "noisy" data from social networks saturated with slang and irony. The gap between 90.98% (training) and 80% (test) indicates moderate overfitting, typical for neural networks without additional regularization.

"The results show that LSTM outperforms conventional machine learning

methods in capturing contextual and sequential aspects of text," the authors note.

Classical algorithms train faster and are easier to interpret, but they process text as a set of independent words — without considering order and context. LSTM, on the other hand, "remembers" previous words when processing each new one: this is why it understands that "not bad" is not "bad," and "didn't like it at all" carries strong negative sentiment.

Why this matters for social media analysis

Twitter (now X) generates hundreds of millions of posts daily. Automatic sentiment analysis allows brands to track audience reaction to their product in real time, and researchers to study the dynamics of public opinion without manually reviewing millions of messages.

Modern commercial systems rely on transformer models — BERT, RoBERTa — which significantly outperform LSTM but require serious computational resources. LSTM remains a practical choice for teams with limited budgets: it is lighter, faster in inference, and fully functional without a powerful GPU cluster.

What this means

The work confirms a persistent pattern in NLP: for text where word order and context matter, deep learning consistently wins over classical ML. Reproducing the experiment is straightforward — an open dataset from Kaggle and standard libraries (TensorFlow, PyTorch) allow you to run a similar pipeline in just a few hours.

Frequently Asked Questions

How does LSTM differ from BERT in sentiment analysis tasks?

LSTM processes text sequentially — word by word, maintaining previous context in memory. BERT is a transformer model that analyzes the entire text at once in both directions and is pretrained on massive corpora. On most modern benchmarks, BERT outperforms LSTM by 5–15 percentage points, but requires significantly more computational resources.

What is ROC-AUC and why is 0.92 a good result?

ROC-AUC shows how well a model separates classes: 0.5 is random guessing, 1.0 is perfect classification. A value of 0.92 is considered high for sentiment analysis tasks on social networks, where data is noisy with slang, typos, and sarcasm.