Processing large amounts of data in Python for the purposes of applied socio-economic research: advantages and current issues

The article is  devoted to  the study of   the features of   processing large amounts of   data using the Python programming language. Unlike tabular processors or   finished software products, programming languages offer the user a   flexible toolkit for the implementation of   tasks. At   the same time, this creates certain risks associated with the effectiveness of  using appropriate tools and optimising the operation of   the programme. The purpose of   the article is   to study the features of   processing large amounts of   data in   Python on  the  examples of   immediate research tasks. The relevance of   the topic and purpose of  the article is   due to   the existing scientific gap related to a  comprehensive consideration of   the technical aspects of   the use of  programming languages and associated tools for socio-economic research. Thus, many authors who use programming languages in   their works rarely provide information regarding the advantages of   certain algorithms or  approaches. Within the framework of  the article, the author examines the procedures and algorithm of   processing a   large array of   data on   the example of   specific research tasks. The conclusions are drawn about the features and advantages of   Python when working with large amounts of   data as   well as  about the prospects for the development of   the relevant scientific topics.

1. Tsypin A.P., Sorokin A.S. Statistical software packages in social and economic researches. Azimut of Scientific Research: Economics and Administration. 2016;4(17(5):379–384. (In Russian).

2. Karpovich A.O. Methods of processing economic information through the Python, programming language. In: XVI Ma-sherov readings: Proceedings of the International Scientific and Practical Conference of Students, Postgraduate Students and Young Scientists, volume 1, Vitebsk, October 21, 2022. Vitebsk: Vitebsk State University named after P.M. Masherov; 2022. Pp. 31–32. (In Russian).

3. Rabinovich A.E., Sedova E.A. Statement of initial analytical hypotheses using data visualization on the example of studying the influence of socio-economic factors on the state of mental health of the population. Russian Economics online journal. 2020;4:44–52. (In Russian).

4. Rusinova M.R. Strategizing the development of territorial socio-economic systems based on the use of “smart” benchmarking tools. Diss. ... Cand. Sci. (Econ.): 08.00.05. Perm: Perm National Research Polytechnic University; 2021. 152 p. (In Russian).

5. Semenyuta O.G., Dubinina I.V., Degtyarev A.S. Improvement of tools to identify key socio-economic factors of influence on the market of housing mortgage lending in Russia. Financial Research. 2021;4(73):106–113. (In Russian).

6. Rungskunroch P., Shen Z.-J., Kaewunruen S. Benchmarking socio-economic impacts of high-speed rail networks using k-nearest neighbour and Pearson’s correlation coefficient techniques through computational model-based analysis. Applied Sciences. 2022,3(12). http://dx.doi.org/10.3390/app12031520

7. Chen M.-Y., Chen T.-H. Modeling public mood and emotion: blog and news sentiment and socio-economic phenomena. Future Generation Computer Systems. 2019;96:692–699. http://dx.doi.org/10.1016/j.future.2017.10.028