Commit graph

103 commits

Author SHA1 Message Date
Andre Medeiros b992d23c03 use a locking mechanism for the scheduler instead of the singleton model in the cron lib 2023-06-08 17:33:22 -04:00
Andre Medeiros b0f7f6fccb have errors for too many requests too 2023-03-16 09:33:51 -04:00
Andre Medeiros d6b967188c things 2022-11-09 14:19:14 -05:00
Andre Medeiros e23d47f815 switch to oauth in subredddit about 2022-11-09 13:59:04 -05:00
Andre Medeiros 038db61798 switch subreddit posts url 2022-11-09 13:54:03 -05:00
Andre Medeiros af538e6f42 update reddit http client 2022-11-05 16:08:00 -04:00
André Medeiros 9341e08123 feat(ops): add distributed tracing (#109)
* tracing

* instrument redis

* add instrumentation around queue acks

* adjust backoff schedules for reddit

* set proper http client on reddit

* populate account last message ID on upsert

* pass last message id on upsert

* schedule accounts in a predictable order
2022-11-05 15:59:33 -04:00
Andre Medeiros 164652e2e4 explicitly use http2 2022-11-02 19:05:23 -04:00
Andre Medeiros b1aac725aa try reddit without verifyign certificates 2022-11-01 19:41:49 -04:00
Andre Medeiros 2c04466a6b more reddit client tweaks 2022-10-26 21:59:44 -04:00
Andre Medeiros 4a805e57d0 experiment with http client changes in reddit 2022-10-26 21:39:04 -04:00
Andre Medeiros bf2dff8497 more precision on measurement 2022-10-26 20:20:45 -04:00
Andre Medeiros f50d9f075e use error maps to define custom errors per request 2022-10-26 18:51:33 -04:00
Andre Medeiros 2b03f4f884 be more forgiving with reddit errors 2022-10-26 18:28:11 -04:00
Andre Medeiros 840d644a4e handle threads with no comments 2022-10-26 13:15:59 -04:00
Andre Medeiros 696f932baa Live activities 2022-10-19 09:37:41 -04:00
Andre Medeiros 4e453e0db2 measure reddit latency when we finish reading the body 2022-07-15 12:36:29 -04:00
Andre Medeiros 102cbb2ee2 setup timeouts on reddit http client 2022-07-15 12:34:28 -04:00
Andre Medeiros f4fea41435 remove some load 2022-07-13 18:43:27 -04:00
Andre Medeiros 556c1dd9b2 test default http client 2022-07-13 16:38:10 -04:00
Andre Medeiros aaf1fbd98a 401 and 403 are forbidden and we'll delete account 2022-07-13 13:09:11 -04:00
Andre Medeiros 9d0c5fb419 allow archived subreddits too 2022-07-12 14:45:19 -04:00
Andre Medeiros e0f194a7d1 allow following restricted subreddits too 2022-07-12 14:37:51 -04:00
Andre Medeiros 655300682f watchers check subreddit info with authentication 2022-06-30 15:24:58 -04:00
Andre Medeiros d5affe36c1 more logging 2022-06-30 15:06:03 -04:00
André Medeiros 8dc4ac350e omg fix watchers and subreddit things (#79)
* omg fix watchers and subreddit things

* don't send thumbnails for posts that are over 18
2022-05-26 17:54:02 -04:00
Andre Medeiros 7da47176a2 Revert "token refresh mechanism"
This reverts commit df96aaa768.
2022-05-25 20:17:03 -04:00
Andre Medeiros df96aaa768 token refresh mechanism 2022-05-25 20:12:54 -04:00
Andre Medeiros 5ede5d6987 more breahing room with tokens 2022-05-25 19:49:14 -04:00
Andre Medeiros ca8fc6e991 tighten authenticated client params 2022-05-25 19:28:41 -04:00
Andre Medeiros dc54bb76df try referencing client instead of embedding 2022-05-25 19:22:05 -04:00
Andre Medeiros b6c7c58062 revert client change 2022-05-23 17:41:19 -04:00
Andre Medeiros 17c503e0e3 don't retry on 403 2022-05-23 17:37:51 -04:00
Andre Medeiros 5fa9bfaa45 distinguish between 401 and 403 2022-05-23 17:26:40 -04:00
Andre Medeiros c35ec006f1 simplify reddit error handling 2022-05-23 11:33:15 -04:00
Andre Medeiros 527552eb6d Use raw_json on all reddit calls 2022-05-17 17:36:59 -04:00
André Medeiros f9b9c595cf Better testing (#62)
* some tests

* more tests

* tidy up go.mod

* more tests

* add postgres

* beep

* again

* Set up schema

* fix device test
2022-05-07 12:37:21 -04:00
Andre Medeiros 4fad10bade fix tests 2022-05-07 11:51:56 -04:00
André Medeiros ead14c8bf0 Merge branch 'main' into chore/schema-changes 2022-05-01 14:07:34 -04:00
Andre Medeiros 9ba4498c08 Add post thumbnails too 2022-05-01 13:33:09 -04:00
Andre Medeiros 7c7e1e5e1c More proper types 2022-03-28 17:33:01 -04:00
Andre Medeiros dbcda74ab8 Changes to schema 2022-03-28 17:05:01 -04:00
Andre Medeiros 294243b02d fix ci 2022-03-26 13:40:51 -04:00
Andre Medeiros fc617b535c fix tests 2022-03-26 13:29:58 -04:00
Andre Medeiros a6abda91a7 add accept header to request json 2022-03-14 09:42:47 -04:00
Andre Medeiros 455788a9c4 strip .json from urls 2022-03-14 09:40:18 -04:00
Andre Medeiros 3bf38b0630 record timestamp and crazy ratelimit cases 2022-03-12 16:00:41 -05:00
Andre Medeiros b9389511f6 log requests per account 2022-03-12 14:46:36 -05:00
Andre Medeiros 195bd55426 more info on ratelimit; change useragent 2022-03-12 14:22:00 -05:00
Andre Medeiros 7bff205523 report rate limit 100% 2022-03-12 14:07:15 -05:00