Commit graph

123 commits

Author SHA1 Message Date
Andre Medeiros
ae764077b4 increase batch size on scheduler 2022-07-13 17:01:04 -04:00
Andre Medeiros
80bae7ff9f get string slice straight from command result 2022-07-13 16:24:51 -04:00
Andre Medeiros
1f59a47880 increase pool size 2022-07-13 15:52:07 -04:00
Andre Medeiros
be7bd24ae1 increase multiplier 2022-07-13 15:30:39 -04:00
Andre Medeiros
b26140f190 test: less prefetch more workers at the same time 2022-07-13 15:08:21 -04:00
Andre Medeiros
a00e4405c7 tweak redis conns 2022-07-13 15:02:38 -04:00
Andre Medeiros
a3e69954c8 up the batch sizes 2022-07-13 12:39:37 -04:00
Andre Medeiros
2fe34ecc99 run in singleton mode 2022-07-13 12:21:12 -04:00
Andre Medeiros
e80579ed35 tweak batch sizes 2022-07-13 12:19:11 -04:00
André Medeiros
e8a8e5a5b3 chore: use reddit ids when enqueueing accounts (#87) 2022-07-13 11:50:13 -04:00
Andre Medeiros
ce944f92f7 lets prune accounts again 2022-06-30 09:56:38 -04:00
André Medeiros
d3c9e303ff Update api.go 2022-06-27 18:47:07 -04:00
Andre Medeiros
523dce6300 tweak pool 2022-06-27 10:36:01 -04:00
Andre Medeiros
8ae426a4de do not prune accounts 2022-06-25 14:45:05 -04:00
Andre Medeiros
f4e5d7e979 bump connections again 2022-06-25 14:04:03 -04:00
Andre Medeiros
aa92f1ec69 increase API connections 2022-06-25 14:01:03 -04:00
Andre Medeiros
221de1d2b6 back to no singleton mode, now that we're stable 2022-06-04 10:29:33 -04:00
Andre Medeiros
d243cebab6 better iteration 2022-06-04 10:24:22 -04:00
Andre Medeiros
084407a5f2 calculate batch size properly 2022-06-04 10:23:45 -04:00
Andre Medeiros
b23a158471 parallelize scheduler better 2022-06-04 10:21:29 -04:00
Andre Medeiros
6ece4c784a no singleton 2022-05-27 16:12:50 -04:00
Andre Medeiros
0938400dd7 Revert "try enqueue accounts in a loop instead"
This reverts commit 9b5829f549.
2022-05-27 15:02:57 -04:00
Andre Medeiros
01ad9f3d1e Revert "another experiment for scheduling"
This reverts commit c8dd76e1ab.
2022-05-27 15:02:47 -04:00
Andre Medeiros
c8dd76e1ab another experiment for scheduling 2022-05-27 14:55:59 -04:00
Andre Medeiros
9b5829f549 try enqueue accounts in a loop instead 2022-05-27 14:47:59 -04:00
Andre Medeiros
fb3f2d1433 more scheduler tweaks 2022-05-27 14:40:40 -04:00
Andre Medeiros
356402d631 Revert "tweak scheduler"
This reverts commit 38fef3c852.
2022-05-27 14:39:12 -04:00
Andre Medeiros
38fef3c852 tweak scheduler 2022-05-27 14:33:46 -04:00
Andre Medeiros
eb853710c4 Add profiling to scheduler 2022-05-27 13:27:28 -04:00
Andre Medeiros
079cc9a5c5 better variable name 2022-05-23 18:54:38 -04:00
Andre Medeiros
34e050daad tweak scheduler 2022-05-23 18:51:30 -04:00
Andre Medeiros
b1ed5fb71a Revert "optimize scheduler to reduce memory allocations"
This reverts commit f41dd925e4.
2022-05-23 18:46:39 -04:00
Andre Medeiros
f41dd925e4 optimize scheduler to reduce memory allocations 2022-05-23 18:43:53 -04:00
Andre Medeiros
bcce102388 more logging things 2022-05-23 14:29:15 -04:00
André Medeiros
b1f266bf91 migrate out of logrus (#76) 2022-05-23 14:17:25 -04:00
Andre Medeiros
6f8f93f3ab reduce account updates by half 2022-05-22 19:57:29 -04:00
Andre Medeiros
b92e3fc374 tweak scheduler 2022-05-22 19:35:03 -04:00
Andre Medeiros
fc94bef379 optimize queries 2022-05-21 17:15:24 -04:00
André Medeiros
f9b9c595cf Better testing (#62)
* some tests

* more tests

* tidy up go.mod

* more tests

* add postgres

* beep

* again

* Set up schema

* fix device test
2022-05-07 12:37:21 -04:00
Andre Medeiros
dbcda74ab8 Changes to schema 2022-03-28 17:05:01 -04:00
Andre Medeiros
5e48ab682b Add app commit to config 2022-03-28 13:50:06 -04:00
Andre Medeiros
7e36609cda right env variable 2022-03-28 13:23:24 -04:00
Andre Medeiros
493f449802 add bugsnag 2022-03-28 13:20:01 -04:00
Andre Medeiros
a1a098b448 ch-ch-changes 2022-03-12 12:50:05 -05:00
Andre Medeiros
70d73eab4c defer func on stats for scheduler 2021-10-17 12:04:09 -04:00
Andre Medeiros
f5d2fe5e81 check for stuck accounts every 2 minutes 2021-10-17 10:18:13 -04:00
Andre Medeiros
c8bec95bed add stuck notifications worker 2021-10-17 10:17:41 -04:00
Andre Medeiros
eb0d6230d7 add tags and validations 2021-10-12 12:04:20 -04:00
Andre Medeiros
65792abd94 add subreddit trending posts watcher 2021-10-10 11:51:42 -04:00
Andre Medeiros
4955f74f52 user following 2021-10-09 10:59:20 -04:00
Andre Medeiros
9854c0ecc4 fix lint 2021-09-27 15:54:29 -04:00
Andre Medeiros
a26b1f6a32 delete devices heuristic fix 2021-09-25 14:56:14 -04:00
Andre Medeiros
3361403379 subreddit notifications 2021-09-25 12:56:01 -04:00
Andre Medeiros
fc1065deb5 fix ci 2021-09-25 09:19:42 -04:00
Andre Medeiros
9a5d699f66 Remove stale devices 2021-08-14 12:08:17 -04:00
Andre Medeiros
84e499a7af schedule stale and orphaned accounts 2021-08-14 11:59:13 -04:00
Andre Medeiros
f6e05a6fd8 clean orphaned accounts 2021-08-14 11:54:48 -04:00
Andre Medeiros
1249f54bf2 tweak accounts and batch size 2021-07-22 21:52:29 -04:00
Andre Medeiros
5848d9584d do 2000 accounts at a time 2021-07-22 21:42:06 -04:00
Andre Medeiros
424e27b068 run scheduler 5 times per second 2021-07-22 21:35:11 -04:00
Andre Medeiros
cbcab2a992 a few more connections please 2021-07-22 21:16:41 -04:00
Andre Medeiros
7692971b89 pool size doesn't need to be that big 2021-07-22 21:11:27 -04:00
Andre Medeiros
e0579fbcbd fix up for render 2021-07-22 20:22:46 -04:00
Andre Medeiros
71f04afb8a tweak scheduler to only run one enqueue at a time 2021-07-20 14:39:01 -04:00
Andre Medeiros
04d59f280d tweak connections on scheduler 2021-07-20 13:00:53 -04:00
Andre Medeiros
ac4765f813 tweak client settings 2021-07-15 20:50:32 -04:00
Andre Medeiros
26ca63f31b add a profile option to the commands 2021-07-15 20:09:38 -04:00
Andre Medeiros
e46082949a tweak pool args 2021-07-15 13:47:22 -04:00
Andre Medeiros
8a777d2ee6 timeout reddit and increase batches 2021-07-13 20:16:38 -04:00
Andre Medeiros
55c45592ac tweak params 2021-07-13 20:09:44 -04:00
Andre Medeiros
bb4b6f0e83 add heroku metrics back 2021-07-13 15:35:46 -04:00
Andre Medeiros
c2e3b81830 Add some debugging 2021-07-13 14:22:41 -04:00
Andre Medeiros
9fa0257d84 reorganize commands to keep things more consistent 2021-07-13 13:14:25 -04:00