touch foxglove NOW hey bot-fans, you may have noticed that fan-favorite bot @autodrama stopped posting about three months ago. This is because the monkeys at pushshift screwed up the way that comments are retrieved. I call into pushshift's API here to get comments on a post, however, this randomly stopped working.
PMAW is a python wrapper for pushshift. I only use it once, and that is to get all the comments on a particular post. The API documentation says that I can do this with "link_id" (https://pushshift.io/api-parameters/) but when I try to hit api.pushshift.io/reddit/comment/search/?link_id=10pv7qa&subreddit=ScienceUncensored, I get this response:
{
"data": [],
"error": null,
"errors": {
"error": {
"root_cause": [
{
"type": "query_shard_exception",
"reason": "failed to create query: For input string: \"10pv7qa\"",
"index_uuid": "htpvQm2RT4uCWzM53Q3zuw",
"index": "rc_2005-12"
},
<above object 200 more times>
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "rc_2005-12",
"node": "r5briY5hS_mr4RzlALKdow",
"reason": {
"type": "query_shard_exception",
"reason": "failed to create query: For input string: \"10pv7qa\"",
"index_uuid": "htpvQm2RT4uCWzM53Q3zuw",
"index": "rc_2005-12",
"caused_by": {
"type": "number_format_exception",
"reason": "For input string: \"10pv7qa\""
}
}
},
<above object 200 more times>
]
},
"status": 400
}
}
Yes, I reported this issue to the maintainers of pushshift, but have gotten no response back.
Clearly, the backend is interpretting what should be strings as numbers, because this works: api.pushshift.io/reddit/comment/search/?link_id=100000&subreddit=trees
I also tried using pushshift's /comment_ids endpoint but shit's broken too.
Fellas, any thoughts? @automeme is my only child that is not estranged, unlike my other children
@bbbb (who hates me 😭) and
@automeme (who is clinically retarded)
Jump in the discussion.
No email address required.
https://old.reddit.com/r/pushshift/comments/103k1qe/anyone_have_luck_using_the_link_id_param_in_the/j2zyjkp/
Jump in the discussion.
No email address required.
unfortunately, same result for newer posts as I described here, with incomplete results. but, I am glad other people know about this!
Jump in the discussion.
No email address required.
Yeah, the comment addresses this at the end. I don't think there's anything you can do about it. The maintainer of the API seems to be in the middle of reindexing his ES cluster.
Jump in the discussion.
No email address required.
codechads, its a good day
Jump in the discussion.
No email address required.
More options
Context
More options
Context
More options
Context
interesting!!!! let me give this a try
Jump in the discussion.
No email address required.
More options
Context
Since when Python had
baseas a second parameter forint()? TIL.Jump in the discussion.
No email address required.
More options
Context
More options
Context