{"topic":{"id":13116,"title":"【求助】使用 Jupyter 学习 MachineLearning 时遇见异常","slug":"topic","categoryId":54,"createdAt":"2024-07-23 15:34:44.189897","bumpedAt":"2024-07-25 05:14:12.295687","views":261,"postsCount":8,"highestPostNumber":8,"likeCount":0,"closed":false,"pinned":false,"pinnedGlobally":false,"pinnedAt":null,"pinnedUntil":null,"isBanner":false,"excerpt":"在使用 jupyter 做深度学习作业时的时候出现了如下错误 File \"C:\\Users\\Administrator\\anaconda3\\Lib\\multiprocessing\\spawn.py\", line 122, in spawn_main exitcode = _main(fd, parent_sentinel) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File \"C:\\Users\\Administrator\\a…","deletedAt":null,"isAnonymous":false,"category":{"id":54,"name":"学术专区","slug":"academic","color":"9EB83B"},"author":{"id":5312,"username":"Snowindly","name":"钰雪泠风","avatarUrl":"/uploads/image/0f/06/0f069e51c023bcc7a48f5f16311a8e5181745bb35bbdb31c8df6451a9fe78857.avif"},"tags":[]},"posts":[{"id":37746,"postNumber":1,"raw":"在使用 jupyter 做深度学习作业时的时候出现了如下错误\n\n File \"C:\\Users\\Administrator\\anaconda3\\Lib\\multiprocessing\\spawn.py\", line 122, in spawn_main\n exitcode = _main(fd, parent_sentinel)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"C:\\Users\\Administrator\\anaconda3\\Lib\\multiprocessing\\spawn.py\", line 132, in _main\n self = reduction.pickle.load(from_parent)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAttributeError: Can't get attribute 'MyDataset' on \n\n经查询疑似是在 windows 系统上使用多进程时报错 AttributeError，但代码内并未使用 multiprocessing，不知是否是 numworker 导致的错误\n\n代码参考 [李宏毅深度学习 2021 春 HW4 实验四，Strong Baseline，使用 Conformer_李宏毅老师深度学习语音预测 hw 如何 change transformer to conformer-CSDN 博客](https://blog.csdn.net/qq_43319080/article/details/120256218?spm=1001.2101.3001.6650.3&utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromBaidu%7ERate-3-120256218-blog-122799554.235%5Ev43%5Epc_blog_bottom_relevance_base2&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7EBlogCommendFromBaidu%7ERate-3-120256218-blog-122799554.235%5Ev43%5Epc_blog_bottom_relevance_base2&utm_relevant_index=6) 的源码","cooked":"

在使用 jupyter 做深度学习作业时的时候出现了如下错误

File \"C:\\Users\\Administrator\\anaconda3\\Lib\\multiprocessing\\spawn.py\", line 122, in spawn_main
\nexitcode = _main(fd, parent_sentinel)
\n^^^^^^^^^^^^^^^^^^^^^^^^^^
\nFile \"C:\\Users\\Administrator\\anaconda3\\Lib\\multiprocessing\\spawn.py\", line 132, in _main
\nself = reduction.pickle.load(from_parent)
\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\nAttributeError: Can't get attribute 'MyDataset' on <module 'main' (built-in)>

经查询疑似是在 windows 系统上使用多进程时报错 AttributeError，但代码内并未使用 multiprocessing，不知是否是 numworker 导致的错误

代码参考李宏毅深度学习 2021 春 HW4 实验四，Strong Baseline，使用 Conformer_李宏毅老师深度学习语音预测 hw 如何 change transformer to conformer-CSDN 博客的源码

\n","createdAt":"2024-07-23 15:34:44.320554","updatedAt":"2024-07-23 15:38:23.157504","replyToPostNumber":null,"replyCount":0,"likeCount":0,"wordCount":141,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":2,"publicVersion":2,"deletedAt":null,"isAnonymous":false,"user":{"id":5312,"username":"Snowindly","name":"钰雪泠风","avatarUrl":"/uploads/image/0f/06/0f069e51c023bcc7a48f5f16311a8e5181745bb35bbdb31c8df6451a9fe78857.avif","admin":false,"moderator":false,"signatureRaw":null,"signatureCooked":null}},{"id":37747,"postNumber":2,"raw":"PS：直接运行助教给的 simple baseline 的代码也出现同样报错","cooked":"

PS：直接运行助教给的 simple baseline 的代码也出现同样报错

\n","createdAt":"2024-07-23 15:39:49.650344","updatedAt":"2024-07-23 15:39:49.650344","replyToPostNumber":null,"replyCount":0,"likeCount":0,"wordCount":21,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"isAnonymous":false,"user":{"id":5312,"username":"Snowindly","name":"钰雪泠风","avatarUrl":"/uploads/image/0f/06/0f069e51c023bcc7a48f5f16311a8e5181745bb35bbdb31c8df6451a9fe78857.avif","admin":false,"moderator":false,"signatureRaw":null,"signatureCooked":null}},{"id":37751,"postNumber":3,"raw":"[quote=\"钰雪泠风，post:1, topic:13116, username:Snowindly\"]\n但代码内并未使用 multiprocessing\n[/quote]\n\n看错误的文件路径：\n\n```\nFile“C:\\Users\\Administrator\\anaconda3\\Lib\\multiprocessing\\spawn.py”, line 122, in spawn_main\n```\n\n显然是间接调用了 `multiprocessing` 的。我比较怀疑是 Windows 下 Python 创建子进程时地址空间的问题。\n\n`multiprocessing` 模块官方文档应该有提到，Windows 下创建进程的方式是 spawn 而不是 fork，子进程不继承父进程地址空间，也许 get attribute 正是因此失败的。如果真是这个原因的话，由于 Windows 平台 Python 不支持 fork 创建子进程，我的建议是换到 POSIX 平台上运行。","cooked":"

\n\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t

\n\t\t\t\t\t\t\t钰雪泠风，post:1, topic:13116, username:Snowindly:\n\t\t\t\t\t\t

\n\t\t\t\t\t

\n
但代码内并未使用 multiprocessing
\n

\n\t\t\t\t

看错误的文件路径：

File“C:\\Users\\Administrator\\anaconda3\\Lib\\multiprocessing\\spawn.py”, line 122, in spawn_main\n

显然是间接调用了 multiprocessing 的。我比较怀疑是 Windows 下 Python 创建子进程时地址空间的问题。

multiprocessing 模块官方文档应该有提到，Windows 下创建进程的方式是 spawn 而不是 fork，子进程不继承父进程地址空间，也许 get attribute 正是因此失败的。如果真是这个原因的话，由于 Windows 平台 Python 不支持 fork 创建子进程，我的建议是换到 POSIX 平台上运行。

\n","createdAt":"2024-07-23 16:21:26.527646","updatedAt":"2024-07-23 16:23:28.196531","replyToPostNumber":null,"replyCount":0,"likeCount":0,"wordCount":162,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"isAnonymous":false,"user":{"id":1381,"username":"greyishsong","name":"greyishsong","avatarUrl":"/uploads/image/da/bc/dabc731b0311f59850625bbeb55ed969fcfac0430c5ef0f68441e5466b7aa496.avif","admin":false,"moderator":true,"signatureRaw":null,"signatureCooked":null}},{"id":37756,"postNumber":4,"raw":"![undefined|690x245](/uploads/image/16/cc/16cc4d1494ead76e048e91527ba4274c90d494cfdd3fdc4e754fda740c8631fc.avif)","cooked":"

$\"undefined\"$

\n","createdAt":"2024-07-23 18:10:06.175794","updatedAt":"2024-07-23 18:10:06.175794","replyToPostNumber":null,"replyCount":0,"likeCount":0,"wordCount":1,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"isAnonymous":false,"user":{"id":5312,"username":"Snowindly","name":"钰雪泠风","avatarUrl":"/uploads/image/0f/06/0f069e51c023bcc7a48f5f16311a8e5181745bb35bbdb31c8df6451a9fe78857.avif","admin":false,"moderator":false,"signatureRaw":null,"signatureCooked":null}},{"id":37759,"postNumber":5,"raw":"经实践，用 pycharm 直接运行代码可行，但是仍未找到如何在 jupyter 上运行的方法","cooked":"

经实践，用 pycharm 直接运行代码可行，但是仍未找到如何在 jupyter 上运行的方法

\n","createdAt":"2024-07-23 18:11:15.559754","updatedAt":"2024-07-23 18:11:15.559754","replyToPostNumber":null,"replyCount":2,"likeCount":0,"wordCount":31,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"isAnonymous":false,"user":{"id":5312,"username":"Snowindly","name":"钰雪泠风","avatarUrl":"/uploads/image/0f/06/0f069e51c023bcc7a48f5f16311a8e5181745bb35bbdb31c8df6451a9fe78857.avif","admin":false,"moderator":false,"signatureRaw":null,"signatureCooked":null}},{"id":37760,"postNumber":6,"raw":"futurewarning 让人看着很难受","cooked":"

futurewarning 让人看着很难受

\n","createdAt":"2024-07-23 18:13:36.217312","updatedAt":"2024-07-23 18:13:36.217312","replyToPostNumber":5,"replyCount":1,"likeCount":0,"wordCount":8,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"isAnonymous":false,"user":{"id":5312,"username":"Snowindly","name":"钰雪泠风","avatarUrl":"/uploads/image/0f/06/0f069e51c023bcc7a48f5f16311a8e5181745bb35bbdb31c8df6451a9fe78857.avif","admin":false,"moderator":false,"signatureRaw":null,"signatureCooked":null}},{"id":37818,"postNumber":7,"raw":"说明 codebase 太老了，应该降级依赖","cooked":"

说明 codebase 太老了，应该降级依赖

\n","createdAt":"2024-07-25 05:06:37.082433","updatedAt":"2024-07-25 05:06:37.082433","replyToPostNumber":6,"replyCount":0,"likeCount":0,"wordCount":13,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"isAnonymous":false,"user":{"id":1277,"username":"ucw","name":null,"avatarUrl":"/uploads/image/03/ea/03eac2176be50969b549f432e72df28bd719033042da4c6c41c3fde705e8f13a.avif","admin":false,"moderator":false,"signatureRaw":null,"signatureCooked":null}},{"id":37819,"postNumber":8,"raw":"maybe related to: https://github.com/Lightning-AI/pytorch-lightning/discussions/15350","cooked":"

\n","createdAt":"2024-07-25 05:14:12.295687","updatedAt":"2024-07-25 05:14:12.295687","replyToPostNumber":5,"replyCount":0,"likeCount":0,"wordCount":4,"liked":false,"bookmarked":false,"polls":[],"reactions":[],"wiki":false,"version":1,"publicVersion":1,"deletedAt":null,"isAnonymous":false,"user":{"id":1277,"username":"ucw","name":null,"avatarUrl":"/uploads/image/03/ea/03eac2176be50969b549f432e72df28bd719033042da4c6c41c3fde705e8f13a.avif","admin":false,"moderator":false,"signatureRaw":null,"signatureCooked":null}}],"hasMore":false,"lastReadPostNumber":7,"hasUrlPostNumber":true}