On 23 Mar 2023, at 08:46, Arnaud Loonstra <[email protected]> wrote:
Hi all,
I'm running in a crash due to a ResourceWarning (some socket is not closed in a used module) after calling PyGILState_Release.
I'm running Python in a native thread (so a thread created by C not Python). I'm acquiring the GIL through PyGILState_Ensure before doing any CPYthon calls and releasing when finished using PyGILState_Release.
This has worked fine. But now I'm using a python module (openai) which doesn't close its socket correctly which results in a ResourceWarning which triggers an assert.
In the backtrace (below) I can see PyGILState_Release is called again. (7) while I've already called it (126).
I can make the crash go away by adding
import warnings
warnings.simplefilter("ignore", ResourceWarning)
to my python code. But I'd rather prevent this from happening in the first place.
Any suggestion very welcomed cause I'm puzzled.
Rg,
Arnaud
1 __pthread_kill_implementation pthread_kill.c 44 0x7ffff6fcaccc
2 __pthread_kill_internal pthread_kill.c 78 0x7ffff6fcad2f
3 __GI_raise raise.c 26 0x7ffff6f7bef2
4 __GI_abort abort.c 79 0x7ffff6f66472
5 __assert_fail_base assert.c 92 0x7ffff6f66395
6 __GI___assert_fail assert.c 101 0x7ffff6f74df2
7 PyGILState_Release pystate.c 1742 0x7ffff7abcf9f
8 tracemalloc_raw_alloc _tracemalloc.c 779 0x7ffff7afa803
9 tracemalloc_raw_malloc _tracemalloc.c 789 0x7ffff7afa844
10 PyMem_RawMalloc obmalloc.c 586 0x7ffff79dab8d
11 decode_current_locale fileutils.c 472 0x7ffff7ada4ab
12 _Py_DecodeLocaleEx fileutils.c 598 0x7ffff7adafce
13 unicode_decode_locale unicodeobject.c 3970 0x7ffff7a13b65
14 PyUnicode_DecodeLocale unicodeobject.c 4012 0x7ffff7a18c79
15 PyErr_SetFromErrnoWithFilenameObjects errors.c 772 0x7ffff7a94178
16 PyErr_SetFromErrnoWithFilenameObject errors.c 750 0x7ffff7a942ce
17 posix_path_object_error posixmodule.c 1709 0x7ffff7ae4201
18 path_object_error posixmodule.c 1719 0x7ffff7ae420f
19 path_error posixmodule.c 1737 0x7ffff7ae4221
20 posix_do_stat posixmodule.c 2565 0x7ffff7af42f7
21 os_stat_impl posixmodule.c 2897 0x7ffff7af4491
22 os_stat posixmodule.c.h 71 0x7ffff7af45e5
23 cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c 443 0x7ffff79d2ece
24 _PyObject_VectorcallTstate pycore_call.h 92 0x7ffff798b579
25 PyObject_Vectorcall call.c 299 0x7ffff798b65e
26 _PyEval_EvalFrameDefault ceval.c 4772 0x7ffff7a75453
27 _PyEval_EvalFrame pycore_ceval.h 73 0x7ffff7a79887
28 _PyEval_Vector ceval.c 6435 0x7ffff7a799b1
29 _PyFunction_Vectorcall call.c 393 0x7ffff798b215
30 _PyObject_VectorcallTstate pycore_call.h 92 0x7ffff798b579
31 PyObject_CallOneArg call.c 376 0x7ffff798b6cd
32 call_show_warning _warnings.c 609 0x7ffff7a31d24
33 warn_explicit _warnings.c 746 0x7ffff7a3223b
34 do_warn _warnings.c 947 0x7ffff7a3236f
35 warn_unicode _warnings.c 1106 0x7ffff7a32414
36 _PyErr_WarnFormatV _warnings.c 1126 0x7ffff7a32481
37 PyErr_ResourceWarning _warnings.c 1177 0x7ffff7a32d38
38 sock_finalize socketmodule.c 5073 0x7ffff6840cf3
39 PyObject_CallFinalizer object.c 208 0x7ffff79d6449
40 PyObject_CallFinalizerFromDealloc object.c 226 0x7ffff79d75ac
41 subtype_dealloc typeobject.c 1382 0x7ffff79ecf81
42 _Py_Dealloc object.c 2390 0x7ffff79d6c38
43 Py_DECREF object.h 527 0x7ffff79c116e
44 Py_XDECREF object.h 602 0x7ffff79c118d
45 free_keys_object dictobject.c 664 0x7ffff79c120a
46 dictkeys_decref dictobject.c 324 0x7ffff79c130f
47 dict_dealloc dictobject.c 2378 0x7ffff79c2eeb
48 _Py_Dealloc object.c 2390 0x7ffff79d6c38
49 Py_DECREF object.h 527 0x7ffff79eb06b
50 subtype_dealloc typeobject.c 1438 0x7ffff79ed138
51 _Py_Dealloc object.c 2390 0x7ffff79d6c38
52 Py_DECREF object.h 527 0x7ffff7affa77
53 deque_clear _collectionsmodule.c 625 0x7ffff7b01e10
54 deque_dealloc _collectionsmodule.c 1311 0x7ffff7b030d9
55 _Py_Dealloc object.c 2390 0x7ffff79d6c38
56 Py_DECREF object.h 527 0x7ffff79c116e
57 Py_XDECREF object.h 602 0x7ffff79c118d
58 _PyObject_FreeInstanceAttributes dictobject.c 5583 0x7ffff79ca68a
59 subtype_dealloc typeobject.c 1442 0x7ffff79ed18d
60 _Py_Dealloc object.c 2390 0x7ffff79d6c38
61 Py_DECREF object.h 527 0x7ffff79c116e
62 Py_XDECREF object.h 602 0x7ffff79c118d
63 _PyObject_FreeInstanceAttributes dictobject.c 5583 0x7ffff79ca68a
64 subtype_dealloc typeobject.c 1442 0x7ffff79ed18d
65 _Py_Dealloc object.c 2390 0x7ffff79d6c38
66 Py_DECREF object.h 527 0x7ffff79c116e
67 Py_XDECREF object.h 602 0x7ffff79c118d
68 free_keys_object dictobject.c 672 0x7ffff79c127f
69 dictkeys_decref dictobject.c 324 0x7ffff79c130f
70 dict_dealloc dictobject.c 2378 0x7ffff79c2eeb
71 odict_dealloc odictobject.c 1364 0x7ffff79ccc8c
72 _Py_Dealloc object.c 2390 0x7ffff79d6c38
73 Py_DECREF object.h 527 0x7ffff79c116e
74 Py_XDECREF object.h 602 0x7ffff79c118d
75 _PyObject_FreeInstanceAttributes dictobject.c 5583 0x7ffff79ca68a
76 subtype_dealloc typeobject.c 1442 0x7ffff79ed18d
77 _Py_Dealloc object.c 2390 0x7ffff79d6c38
78 Py_DECREF object.h 527 0x7ffff79c116e
79 Py_XDECREF object.h 602 0x7ffff79c118d
80 _PyObject_FreeInstanceAttributes dictobject.c 5583 0x7ffff79ca68a
81 subtype_dealloc typeobject.c 1442 0x7ffff79ed18d
82 _Py_Dealloc object.c 2390 0x7ffff79d6c38
83 Py_DECREF object.h 527 0x7ffff79c116e
84 Py_XDECREF object.h 602 0x7ffff79c118d
85 _PyObject_FreeInstanceAttributes dictobject.c 5583 0x7ffff79ca68a
86 subtype_dealloc typeobject.c 1442 0x7ffff79ed18d
87 _Py_Dealloc object.c 2390 0x7ffff79d6c38
88 Py_DECREF object.h 527 0x7ffff79c116e
89 Py_XDECREF object.h 602 0x7ffff79c118d
90 free_keys_object dictobject.c 664 0x7ffff79c120a
91 dictkeys_decref dictobject.c 324 0x7ffff79c130f
92 dict_dealloc dictobject.c 2378 0x7ffff79c2eeb
93 odict_dealloc odictobject.c 1364 0x7ffff79ccc8c
94 _Py_Dealloc object.c 2390 0x7ffff79d6c38
95 Py_DECREF object.h 527 0x7ffff79c116e
96 Py_XDECREF object.h 602 0x7ffff79c118d
97 _PyObject_FreeInstanceAttributes dictobject.c 5583 0x7ffff79ca68a
98 subtype_dealloc typeobject.c 1442 0x7ffff79ed18d
99 _Py_Dealloc object.c 2390 0x7ffff79d6c38
100 Py_DECREF object.h 527 0x7ffff79c116e
101 Py_XDECREF object.h 602 0x7ffff79c118d
102 free_keys_object dictobject.c 664 0x7ffff79c120a
103 dictkeys_decref dictobject.c 324 0x7ffff79c130f
104 dict_dealloc dictobject.c 2378 0x7ffff79c2eeb
105 _Py_Dealloc object.c 2390 0x7ffff79d6c38
106 Py_DECREF object.h 527 0x7ffff79c116e
107 delitem_common dictobject.c 1986 0x7ffff79c4deb
108 _PyDict_DelItem_KnownHash dictobject.c 2028 0x7ffff79c6c8b
109 PyDict_DelItem dictobject.c 2003 0x7ffff79c6d3a
110 _localdummy_destroyed _threadmodule.c 1044 0x7ffff7b31a85
111 cfunction_vectorcall_O methodobject.c 514 0x7ffff79d30f5
112 _PyObject_VectorcallTstate pycore_call.h 92 0x7ffff798b579
113 PyObject_CallOneArg call.c 376 0x7ffff798b6cd
114 handle_callback weakrefobject.c 931 0x7ffff7a300ab
115 PyObject_ClearWeakRefs weakrefobject.c 977 0x7ffff7a305cd
116 localdummy_dealloc _threadmodule.c 673 0x7ffff7b31f18
117 _Py_Dealloc object.c 2390 0x7ffff79d6c38
118 Py_DECREF object.h 527 0x7ffff79c116e
119 Py_XDECREF object.h 602 0x7ffff79c118d
120 free_keys_object dictobject.c 664 0x7ffff79c120a
121 dictkeys_decref dictobject.c 324 0x7ffff79c130f
122 dict_dealloc dictobject.c 2378 0x7ffff79c2eeb
123 _Py_Dealloc object.c 2390 0x7ffff79d6c38
124 Py_DECREF object.h 527 0x7ffff7aba1c9
125 PyThreadState_Clear pystate.c 1029 0x7ffff7abc790
126 PyGILState_Release pystate.c 1743 0x7ffff7abcef3
127 pythonactor_socket pythonactor.c 1021 0x555555709488
128 pythonactor_handle_msg pythonactor.c 1113 0x5555557098a5
129 pythonactor_handler pythonactor.c 1077 0x555555709716
130 sphactor_actor_run_once sphactor_actor.c 965 0x55555599bdfa
131 sphactor_actor_run sphactor_actor.c 1052 0x55555599c1c0
132 s_thread_shim zactor.c 68 0x5555559a97de
133 start_thread pthread_create.c 442 0x7ffff6fc8fd4
134 clone3 clone3.S 81 0x7ffff704966c
--
https://mail.python.org/mailman/listinfo/python-list
On 23 Mar 2023, at 08:46, Arnaud Loonstra <[email protected]> wrote:
Hi all,
I'm running in a crash due to a ResourceWarning (some socket is not closed in a used module) after calling PyGILState_Release.
I'm running Python in a native thread (so a thread created by C not Python). I'm acquiring the GIL through PyGILState_Ensure before doing any CPYthon calls and releasing when finished using PyGILState_Release.
This has worked fine. But now I'm using a python module (openai) which doesn't close its socket correctly which results in a ResourceWarning which triggers an assert.
In the backtrace (below) I can see PyGILState_Release is called again. (7) while I've already called it (126).
I can make the crash go away by adding
import warnings
warnings.simplefilter("ignore", ResourceWarning)
to my python code. But I'd rather prevent this from happening in the first place.
Any suggestion very welcomed cause I'm puzzled.
What 3rd party C extension are you running with?
I'd be surprised if the cpython code was the issue.
Barry
On 23 Mar 2023, at 14:34, Arnaud Loonstra <[email protected]> wrote:
On 23-03-2023 13:33, Barry Scott wrote:
What 3rd party C extension are you running with?On 23 Mar 2023, at 08:46, Arnaud Loonstra <[email protected]> wrote:
Hi all,
I'm running in a crash due to a ResourceWarning (some socket is not closed in a used module) after calling PyGILState_Release.
I'm running Python in a native thread (so a thread created by C not Python). I'm acquiring the GIL through PyGILState_Ensure before doing any CPYthon calls and releasing when finished using PyGILState_Release.
This has worked fine. But now I'm using a python module (openai) which doesn't close its socket correctly which results in a ResourceWarning which triggers an assert.
In the backtrace (below) I can see PyGILState_Release is called again. (7) while I've already called it (126).
I can make the crash go away by adding
import warnings
warnings.simplefilter("ignore", ResourceWarning)
to my python code. But I'd rather prevent this from happening in the first place.
Any suggestion very welcomed cause I'm puzzled.
I'd be surprised if the cpython code was the issue.
Barry
I'm not using any 3rd party extension myself. But the issue is caused by the openai module using the requests module which is not closing sockets:
https://github.com/openai/openai-python/issues/140
I'm not aware what C extensions they might use.
Btw, I've tested this with python 3.8 and 3.11.2.
Rg,
Arnaud
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 715 |
| Nodes: | 16 (3 / 13) |
| Uptime: | 142:47:57 |
| Calls: | 12,089 |
| Calls today: | 2 |
| Files: | 14,998 |
| Messages: | 6,517,455 |