Keywords
AI agents
Political commemorations
Berlin Wall
Day of Europe
large language models
How to Cite
Abstract
Large language models have enabled large-scale analysis of many phenomena, including political discourse on social media. We analyze how finetuning models on social media posts can be used in discourse analysis. We first present a theoretical framework for analyzing political discourse and show that finetuned models are better at detecting discourse quality. We finetune models on examples that match specific discourse quality indicators and demonstrate how this process can align messages with the desired indicator.
References
Achiam, Josh, Adler, Steven, Agarwal, Sandhini, Ahmad, Lama, Akkaya, Ilge, Aleman, Florencia Leoni et al. (2023): GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (last access: 2026-06-23).
Barbera, Pablo (2015): Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data. Political Analysis, 23, 1, 76–91.
Bachtiger, Andre, Shikano, Susumu, Pedrini, Seraina & Mirjam Ryser (2009): Measuring Deliberation 2.0: Standards, Discourse Types, and Sequenzialization. In: ECPR General Conference. Potsdam, 5–12.
Beauchamp, Nick (2020): Modeling and Measuring Deliberation Online. In: Foucault Welles, Brooke & Sandra González-Bailón (eds.): The Oxford Handbook of Networked Communication. Oxford, Oxford University Press, 321–349.
Boyd, Danah (2011): Social Network Sites as Networked Publics: Affordances, Dynamics, and Implications. In: Papacharissi, Zizi (ed.): A Networked Self: Identity, Community, and Culture on Social Network Sites. New York, Routledge, 39–58.
Brown, Tom B., Mann, Benjamin, Ryder, Nick, Subbiah, Melanie, Kaplan, Jared D., Dhariwal, Prafulla et al. (2020): Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
Canute, Matt, Jin, Mali, Holtzclaw, Hannah, Lusoli, Alberto, Adams, Philippa, Pandya, Mugdha, Taboada, Maite, Maynard, Diana & Wendy Hui Kyong Chun (2023): Dimensions of Online Conflict: Towards Modeling Agonism. In: Findings of the Association for Computational Linguistics: EMNLP 2023. Singapore, Association for Computational Linguistics, 12194–12209.
Casper, Stephen, Bailey, Luke, Hunter, Rosco, Ezell, Carson, Cabale, Emma, Gerovitch, Michael, Slocum, Stewart, Wei, Kevin, Jurkovic, Nikola, Khan, Ariba, Christoffersen, Phillip J. K., Ozisik, A. Pinar, Trivedi, Rakshit, Hadfield-Menell, Dylan & Noam Kolt (2025): The AI Agent Index. arXiv preprint arXiv:2502.01635 (last access: 2026-06-23).
Cinelli, Matteo, De Francisci Morales, Gianmarco, Galeazzi, Alessandro, Quattrociocchi, Walter & Michele Starnini (2021): The Echo Chamber Effect on Social Media. Proceedings of the National Academy of Sciences, 118, 9, e2023301118.
Coe, Kevin, Kenski, Kate & Stephen A. Rains (2014): Online and Uncivil? Patterns and Determinants of Incivility in Newspaper Website Comments. Journal of Communication, 64, 4, 658–679.
Conover, Michael D., Ratkiewicz, Jacob, Francisco, Matthew, Goncalves, Bruno, Menczer, Filippo & Alessandro Flammini (2011): Political Polarization on Twitter. Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media, 5, 1, 89–96.
Devlin, Jacob, Chang, Ming-Wei, Lee, Kenton & Kristina Toutanova (2018): BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805 (last access: 2026-06-23).
Ferrara, Emilio, Varol, Onur, Davis, Clayton, Menczer, Filippo & Alessandro Flammini (2016): The Rise of Social Bots. Communications of the ACM, 59, 7, 96–104.
Fortuna, Paula & Sergio Nunes (2018): A Survey on Automatic Detection of Hate Speech in Text. ACM Computing Surveys (CSUR), 51, 4, 1–30.
Fournier-Tombs, Eleonore (2024): An Ethical Grey Zone: AI Agents in Political Deliberations. https://carnegiecouncil.org/media/article/ethical-grey-zone-ai-agents-political-deliberation (last access: 2026-06-23).
Fournier-Tombs, Eleonore & Michael K. MacKenzie (2021): Big Data and Democratic Speech: Predicting Deliberative Quality Using Machine Learning Techniques. Methodological Innovations, 14, 2.
Fraser, Nancy (1990): Rethinking the Public Sphere: A Contribution to the Critique of Actually Existing Democracy. Social Text, 25/26, 56–80.
Gillespie, Tarleton (2018): Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. New Haven, Yale University Press.
Grimmer, Justin & Brandon M. Stewart (2013): Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts. Political Analysis, 21, 3, 267–297.
Gutman, Yifat & Jenny Wustenberg (eds.) (2023): The Routledge Handbook of Memory Activism. London, Routledge.
Habermas, Jurgen (1989): The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. Cambridge, Polity.
Habermas, Jurgen (1996): Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. Cambridge, MIT Press.
Horvat, Marjan & Jure Koražija (2026): Europe Day and the Fall of the Berlin Wall on Twitter/X: Conflict, Tone, and Deliberative Quality Across France, Germany, Italy, and Slovenia. https://doi.org/10.5281/zenodo.18770520 (last access: 2026-06-23).
Ilyas, Sanaa & Qamar Khushi (2012): Facebook Status Updates: A Speech Act Analysis. Academic Research International, 3, 2, 500–507.
Iyengar, Shanto, Lelkes, Yphtach, Levendusky, Matthew, Malhotra, Neil & Sean J. Westwood (2019): The Origins and Consequences of Affective Polarization in the United States. Annual Review of Political Science, 22, 1, 129–146.
Jaidka, Kokil (2022): Talking Politics: Building and Validating Data-Driven Lexica to Measure Political Discussion Quality. Computational Communication Research, 4, 2, 486–527.
Jungherr, Andreas & Adrian Rauchfleisch (2025): Artificial Intelligence in Deliberation: The AI Penalty and the Emergence of a New Deliberative Divide. Government Information Quarterly, 42, 4, 102079.
Kamath, Aishwarya, Ferret, Johan, Pathak, Shreya, Vieillard, Nino et al. (2025): Gemma 3 Technical Report. arXiv preprint arXiv:2503.19786 (last access: 2026-06-23).
Lampe, Urška, Horvat, Marjan, Koražija, Jure, Ergaver, Angelika & Darko Darovec (2026): Agonistic Engagement in Memory Politics: Media Arenas, Normative Orientations, and Debates on Giorno del Ricordo in Italy and Slovenia. https://doi.org/10.5281/zenodo.18770931 (last access: 2026-06-23).
Laver, Michael, Benoit, Kenneth & John Garry (2003): Extracting Policy Positions from Political Texts Using Words as Data. American Political Science Review, 97, 2, 311–331.
Liu, Aixin, Feng, Bei, Xue, Bing, Wang, Bingxuan, Wu, Bochao, Lu, Chengda. et al. (2024): DeepSeek-V3 Technical Report. arXiv preprint arXiv:2412.19437 (last access: 2026-06-23).
Mansbridge, Jane, Bohman, James, Chambers, Simone, Christiano, Thomas, Fung, Archon, Parkinson, John & Mark E. Warren (2012): A Systemic Approach to Deliberative Democracy. In: Parkinson, John & Jane Mansbridge (eds.): Deliberative Systems: Deliberative
Democracy at the Large Scale. Cambridge, Cambridge University Press, 1–26.
Marcellino, William, Beauchamp-Mustafaga, Nathan, Kerrigan, Amanda, Chao, Lev Navarre & Jackson Smith (2023): The Rise of Generative AI and the Coming Era of Social Media Manipulation 3.0: Next-Generation Chinese Astroturfing and Coping with Ubiquitous AI. https://www.rand.org/pubs/perspectives/PEA2679-1.html (last access: 2026-06-23).
Marone, Marc, Weller, Orion, Fleshman, William, Yang, Eugene, Lawrie, Dawn & Benjamin Van Durme (2025): mmBERT: A Modern Multilingual Encoder with Annealed Language Learning. arXiv preprint arXiv:2509.06888. https://arxiv.org/abs/2509.06888 (last access: 2026-06-23).
McPherson, Miller, Smith-Lovin, Lynn & James M. Cook (2001): Birds of a Feather: Homophily in Social Networks. Annual Review of Sociology, 27, 1, 415–444.
Meyer, Erik (2008): Memory and Politics. In: Erll, Astrid & Ansgar Nünning (eds.): Cultural Memory Studies: An International and Interdisciplinary Handbook. Berlin, Walter de Gruyter, 173–180.
Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, Corrado, Greg & Jeff Dean (2013): Distributed Representations of Words and Phrases and Their Compositionality. Advances in Neural Information Processing Systems, 26. arXiv preprint arXiv:1310.4546. https://arxiv.org/abs/1310.4546 (last access: 2026-06-23).
OpenAI (2026): Predstavljamo GPT‑5.2. https://openai.com/index/introducing-gpt-5-2/ (last access: 2026-06-23).
Papacharissi, Zizi (2004): Democracy Online: Civility, Politeness, and the Democratic Potential of Online Political Discussion Groups. New Media & Society, 6, 2, 259–283.
Parthasarathy, Venkatesh Balavadhani, Zafar, Ahtsham, Khan, Aafaq & Arsalan Shahid (2024): The Ultimate Guide to Finetuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities. arXiv preprint arXiv:2408.13296. https://arxiv.org/abs/2408.13296 (last access: 2026-06-23).
Russell, Stuart & Peter Norvig (2003): Artificial Intelligence: A Modern Approach. Upper Saddle River, Pearson Education.
Saha, Tulika, Saha, Sriparna & Pushpak Bhattacharyya (2019): Tweet Act Classification: A Deep Learning Based Classifier for Recognizing Speech Acts in Twitter. In: 2019 International Joint Conference on Neural Networks (IJCNN), 1–8.
Schick, Timo, Dwivedi-Yu, Jane, Dessi, Roberto, Raileanu, Roberta, Lomeli, Maria, Zettlemoyer, Luke, Cancedda, Nicola & Thomas Scialom (2023): Toolformer: Language Models Can Teach Themselves to Use Tools. Advances in Neural Information Processing Systems, 36, 68539–68551.
Searle, John R. (1969): Speech Acts: An Essay in the Philosophy of Language. Cambridge, Cambridge University Press.
Steenbergen, Marco R., Bachtiger, Andre, Sporndli, Markus & Jurg Steiner (2003): Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1, 1, 21–48.
Sumers, Theodore R., Yao, Shunyu, Narasimhan, Karthik R. & Thomas L. Griffiths (2024): Cognitive Architectures for Language Agents. Transactions on Machine Learning Research. arXiv preprint arXiv: 2309.02427. https://arxiv.org/abs/2309.02427 (last access: 2026-06-23).
Sunstein, Cass R. (2018): #Republic: Divided Democracy in the Age of Social Media. Princeton, Princeton University Press.
Škvorc, Tadej, Horvat, Marjan, Koražija, Jure, & Robnik-Šikonja, Marko (2026): Appendix: LLM prompt, List of Topics and List of generated posts for „Towards future AI agents for improved political discourse quality with large language models“. Zenodo. https://doi.org/10.5281/zenodo.20817044 (last access: 2026-06-23).
Tessler, Michael Henry, Bakker, Michiel A., Jarrett, Daniel, Sheahan, Hannah, Chadwick, Martin J., Koster, Raphael, Evans, Georgina, Campbell-Gillingham, Lucy, Collins, Tantum, Parkes, David C., Botvinick, Matthew & Christopher Summerfield (2024): AI Can Help Humans Find Common Ground in Democratic Deliberation. Science, 386, 6719.
Touvron, Hugo, Martin, Louis, Stone, Kevin, Albert, Peter, Almahairi, Amjad, Babaei, Yasmine et al. (2023): Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint
arXiv:2307.09288 (last access: 2026-06-23).
Tornberg, Petter (2025): Large Language Models Outperform Expert Coders and Supervised
Classifiers at Annotating Political Social Media Messages. Social Science Computer Review, 43, 6, 1181–1195.
Tucker, Joshua A., Guess, Andrew, Barbera, Pablo, Vaccari, Cristian, Siegel, Alexandra, Sanovich, Sergey, Stukal, Denis & Brendan Nyhan (2018): Social Media, Political Polarization, and Political Disinformation: A Review of the Scientific Literature. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3144139 (last access: 2026-06-23).
Vosoughi, Soroush, Roy, Deb & Sinan Aral
(2018): The Spread of True and False News Online. Science, 359, 6380, 1146–1151.
Wei, Jason, Wang, Xuezhi, Schuurmans, Dale, Bosma, Maarten, Ichter, Brian, Xia, Fei, Chi, Ed, Le, Quoc & Denny Zhou (2022): Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35, 24824–24837.
Weizenbaum, Joseph (1976): Computer Power and Human Reason: From Judgment to Calculation. San Francisco, W. H. Freeman and Company.
Wustenberg, Jenny (2017): Civil Society and Memory in Postwar Germany. Cambridge, Cambridge University Press.
Zhang, Renxian, Dehong Gao & Wenjie Li (2011): What Are Tweeters Doing: Recognizing
Speech Acts in Twitter. Analyzing Microtext: Papers from the 2011 AAAI Workshop (WS-11-05). Palo Alto, AAAI Press, 86–91.
Ziems, Caleb, Held, William, Shaikh, Omar, Chen, Jiaao, Zhang, Zhehao & Diyi Yang (2024): Can Large Language Models Transform Computational Social Science?. Computational Linguistics, 50, 1, 237–291.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright (c) 2026 Tadej Škvorc, Marjan Horvat, Jure Koražija, Marko Robnik-Šikonja