![]() When selecting the number of threads you can follow one of the two methodologies: The optimal number of threads for requests is between 10 and 20 depending on the current response time of the Pushshift server. Increasing the number of threads you use allows you to make more concurrent requests to Pushshift, however, the returns are diminishing as requests are constrained by the rate-limit. This is optional and defaults to 10, however, you should provide a value as this may not be appropriate for your machine. The number of threads to use during multithreading is set with the num_workers parameter. View the optional parameters for PushshiftAPI here. To install it via pip, run:įrom pmaw import PushshiftAPI api = PushshiftAPI() PMAW currently supports Python 3.5 or later. Implementing intelligent rate limiting can ensure that we minimize the number of rejected requests, and the time it takes to complete. Since API requests are I/O-bound they can benefit from being run asynchronously using multiple threads. The time it takes for your code to complete pulling all this data is limited by both your network latency and the response time of the Pushshift server, which can vary throughout the day.Ĭurrent API libraries such as PRAW and PSAW currently run requests sequentially, which can cause thousands of API calls to take many hours to complete. When building large datasets from Reddit submission and comment data it can require thousands of API calls to the Pushshift API. Search Submission Comment IDs: search_submission_comment_ids.The following three methods are currently supported: When using a method PMAW will complete all the required API calls to complete the query before returning a Response generator object. Parameters are provided through keyword arguments when calling the method, some methods will have required parameters. General usage is through the PushshiftAPI class which provides methods for interacting with different Pushshift endpoints, please view the Pushshift Docs for more details on the endpoints and accepted parameters. Enabled by default.PMAW is a wrapper for the Pushshift API which uses multithreading to retrieve Reddit comments and submissions. Added option to make all links point to. Fixed bug that sometimes caused the search results to not render. Thanks nkmathew for the report and debugging help). Fixed bug that caused the popup size/zoom to change when a hyperlink Updated Handlebars.js to latest version to include recently released Minor improvements to the options page. Fix bug in ctrl/cmd/shift/alt+click behavior. Update Handlebars.js to 4.7.7 as per security advisory. Fix bug that caused the popup to close after opening a link using ctrl + click. Can now find even more results (in some cases) by omitting the URL's protocol (" when doing a fuzzy search. Can now find even more results (in some cases) by using the /duplicatesĪPI endpoint. Add option to show comment count instead of post count on extension icon when In case of any issues, suggestion, or general feedback, please feel free to reach out either here or on Github (). If you find this extension useful, please consider leaving a rating and review. Cache results for a (configurable) amount of time to reduce network usage Blacklist sites to exclude them from auto-search Sort results by score, # of comments, age, or subreddit Special YouTube handling: search by video ID to find all variations of YouTube links You can either initiate the search by clicking the extension button, or by having auto-search enabled, which automatically does the search in the background for every URL you visit. ![]() Find discussions of the current URL on Reddit Minimal, light-weight, open-source extension that finds Reddit submission threads of the current URL using Reddit's public API.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |