I’m creating a Django project in which I call threads to do some tasks for me.
all the thread creation code I have is as below:
thread = threading.Thread(target=check_businesses, name="startup_thread")
crawl_thread = threading.Thread(target=crawl,name=f"crawl_thread_{username}",args=[username,POSTS_TO_CRAWL_COUNT,result_queue])
process_thread = threading.Thread(target=process_queue,name=f"procss_thread_{username}",args=[business,result_queue])
p_t = threading.Thread(target=handle_post,name=f"handle_post_{post.pk}",args=(post,))
m_t = threading.Thread(target=handle_media, name=f"handle_media_{m.pk}",args=(m,))
although these are all the threads I create when I do threading.enumerate() it gives me, among others, a thread with the name Thread-<n> (process_request_thread) where n is a number that gets increased everyt ime my startup_task is called. startp_task is the function in which I start startup_thread (check_business funciton) which in turn starts two new threads: crawl_thread and process_thread. Crawl thread starts crawling a website and stores the data in result_queue. process thread then reads items from this queue and stores their data in the database along with sending them to a remote server. The crawled data consists of a business, its posts and the medias of its posts. For sending each of these value I create a new thread.
Also the remote server periodically calls an api which triggers crawling (basically calls startuo_task).
that’s all there is to it.
I don’t know and can’t find out why it creates all these new threads.
heres’ a part of my recenr log:
INFO - 2024-01-03 10:14:00,128 cyrex.tasks: ['MainThread', 'startup_thread', 'django-main-thread', 'procss_thread__yasmin_shop_tj', 'Thread-510 (process_request_thread)']
INFO - 2024-01-03 10:16:00,127 cyrex.tasks: ['MainThread', 'startup_thread', 'django-main-thread', 'procss_thread__yasmin_shop_tj', 'Thread-511 (process_request_thread)']
INFO - 2024-01-03 10:18:00,136 cyrex.tasks: ['MainThread', 'startup_thread', 'django-main-thread', 'procss_thread__yasmin_shop_tj', 'Thread-512 (process_request_thread)']
INFO - 2024-01-03 10:20:00,126 cyrex.tasks: ['MainThread', 'startup_thread', 'django-main-thread', 'procss_thread__yasmin_shop_tj', 'Thread-513 (process_request_thread)']
