GET /api/v2/video/2134
HTTP 200 OK Vary: Accept Content-Type: text/html; charset=utf-8 Allow: GET, PUT, PATCH, HEAD, OPTIONS
{ "category": "SciPy 2013", "language": "English", "slug": "import-without-a-filesystem-scipy-2013-presentat-1", "speakers": [], "tags": [ "Tech" ], "id": 2134, "state": 1, "title": "Import without a filesystem, SciPy 2013 Presentation", "summary": "Presenter: Pat Marion\n\nAuthors: Pat Marion, Kitware; Aron Ahmadia; Bradley M. Froehle, University of California, Berkeley\n\nTrack: General\n\nScientific Python is growing in popularity among HPC and supercomputing communities, but suffers from a seemingly simple and fundamental problem: importing modules from a shared network filesystem at extreme scale will cripple the performance of a parallel Python program.\n\nAt SciPy '12, the presentation titled \"Solving the import problem: Scalable Dynamic Loading Network File Systems\" analyzed the issue and proposed several remedies, but concluded there was more work to be done. Now, this talk introduces a new technique that leverages the linker to embed C-extension modules, and uses Python freeze to embed pure python modules. The result is a program that imports the Python standard library and scientific Python modules such as NumPy without accessing the filesystem. It achieves near-instant, and always-constant, import time even at full machine scale on today's largest supercomputers. The same technique is also relevant to Python app developers on mobile and embedded systems where filesystem access and dynamic loading inflate app startup time.\n\nThis talk will discuss the concepts involved using a simple hello-world demonstration, and overview a real-world example where Python was used to compute at full machine scale on Argonne's Intrepid BlueGene/P supercomputer.", "description": "", "quality_notes": "", "copyright_text": "http://www.youtube.com/t/terms", "embed": "<object width=\"640\" height=\"390\"><param name=\"movie\" value=\"http://youtube.com/v/EOiEIWMYkwE?version=3&amp;hl=en_US\"></param><param name=\"allowFullScreen\" value=\"true\"></param><param name=\"allowscriptaccess\" value=\"always\"></param><embed src=\"http://youtube.com/v/EOiEIWMYkwE?version=3&amp;hl=en_US\" type=\"application/x-shockwave-flash\" width=\"640\" height=\"390\" allowscriptaccess=\"always\" allowfullscreen=\"true\"></embed></object>", "thumbnail_url": "http://i1.ytimg.com/vi/EOiEIWMYkwE/hqdefault.jpg", "duration": null, "video_ogv_length": null, "video_ogv_url": null, "video_ogv_download_only": false, "video_mp4_length": null, "video_mp4_url": null, "video_mp4_download_only": false, "video_webm_length": null, "video_webm_url": null, "video_webm_download_only": false, "video_flv_length": null, "video_flv_url": null, "video_flv_download_only": false, "source_url": "http://www.youtube.com/watch?v=EOiEIWMYkwE", "whiteboard": "needs editing", "recorded": "2013-07-01", "added": "2013-07-04T10:08:59", "updated": "2014-04-08T20:28:26.404" }