63 commits

Author	SHA1	Message	Date
Andre Medeiros	60e18c37f7	be more aggressive with scheduling	2021-07-09 00:27:50 -04:00
Andre Medeiros	961e41094d	set the key only if it doesn't exist, otherwise it will never expire	2021-07-09 00:11:05 -04:00
Andre Medeiros	fb9741f7e0	make sure consumers always have plenty of work to do	2021-07-08 23:56:03 -04:00
Andre Medeiros	5cd3eeab09	fix delivery ack in jobs	2021-07-08 23:50:34 -04:00
Andre Medeiros	b9373832eb	actually remember last message id so we don't duplicate notifications	2021-07-08 23:29:53 -04:00
Andre Medeiros	3ce858927b	do batches in redis	2021-07-08 23:12:50 -04:00
Andre Medeiros	b7aea89cfc	Measure schedule time	2021-07-08 22:15:28 -04:00
Andre Medeiros	d65bab7970	rejig sql	2021-07-08 22:09:14 -04:00
Andre Medeiros	9c70cee166	fix error handling	2021-07-08 21:50:45 -04:00
Andre Medeiros	3d8492e757	change locking around	2021-07-08 21:44:06 -04:00
Andre Medeiros	83fb4cdcc7	increase gomaxprocs	2021-07-08 21:28:43 -04:00
Andre Medeiros	26a034a68b	fix logic again	2021-07-08 21:20:49 -04:00
Andre Medeiros	09c42c379c	set things right	2021-07-08 21:07:01 -04:00
Andre Medeiros	0f70725be5	enqueue by batches	2021-07-08 21:01:45 -04:00
Andre Medeiros	45576b4d16	batch in second intervals	2021-07-08 20:53:09 -04:00
Andre Medeiros	4d2554fba6	bump up workers and poll duration	2021-07-08 20:44:12 -04:00
Andre Medeiros	abf8823cf1	use .Err to invoke the command	2021-07-08 20:37:39 -04:00
Andre Medeiros	f956033e2d	update last checked at	2021-07-08 20:33:35 -04:00
Andre Medeiros	aad2434d83	fix skipped accounts and locks	2021-07-08 20:26:01 -04:00
Andre Medeiros	c1f1d2a309	log info	2021-07-08 20:17:20 -04:00
Andre Medeiros	4719ee091e	clear lock after job succeeds too	2021-07-08 20:14:12 -04:00
Andre Medeiros	eddf7b53d1	fix notification rates	2021-07-08 20:11:46 -04:00
Andre Medeiros	2b9da40594	prioritize by last time accounts were checked	2021-07-08 20:06:22 -04:00
Andre Medeiros	66c4c28cb4	update account enqueueing logic	2021-07-08 20:03:56 -04:00
Andre Medeiros	dea42a40f6	stats where they belong yay	2021-07-08 19:26:15 -04:00
Andre Medeiros	280c7eb593	added new things	2021-07-08 19:03:46 -04:00
Andre Medeiros	8455d8941c	increase connection lifetime	2021-07-07 22:59:19 -04:00
Andre Medeiros	15c543bb62	tweak db connections	2021-07-07 22:58:21 -04:00
Andre Medeiros	e56483dbb7	build up space for GC and main thread/statsd	2021-07-07 22:34:57 -04:00
Andre Medeiros	b55bb4839b	measure reddit connections	2021-07-07 22:19:02 -04:00
Andre Medeiros	9156b7f07d	measure dequeue time	2021-07-07 21:58:36 -04:00
Andre Medeiros	67ae62b765	add heroku metrics	2021-07-07 21:44:52 -04:00
Andre Medeiros	827e61f3f9	change how we select accounts	2021-07-07 18:47:59 -04:00
Andre Medeiros	9b73c74486	fix expiry	2021-07-07 18:46:12 -04:00
Andre Medeiros	809b075804	fix float issue	2021-07-07 18:44:14 -04:00
Andre Medeiros	69e7cfea71	add some precision to last checked	2021-07-07 18:37:10 -04:00
Andre Medeiros	a929de67d4	postgres does not like being passed backoff	2021-07-07 18:12:45 -04:00
Andre Medeiros	ec7bb411fd	measure if we are falling behind	2021-07-07 18:09:29 -04:00
Andre Medeiros	3b9a1ecd3d	bump to 6 threads per cpu	2021-07-07 18:04:24 -04:00
Andre Medeiros	725ad8bf93	release transaction before sleeping in worker	2021-07-07 17:51:27 -04:00
Andre Medeiros	c346b728fc	latency is a positive value	2021-07-07 17:18:58 -04:00
Andre Medeiros	5095b78681	Measure latency properly	2021-07-07 17:15:44 -04:00
Andre Medeiros	87f9adf55b	Scale back threads a bit	2021-07-07 16:30:28 -04:00
Andre Medeiros	c2ea210003	easy on the logs	2021-07-07 15:19:11 -04:00
Andre Medeiros	6bd4981201	sandbox and production clients	2021-07-07 11:37:30 -04:00
Andre Medeiros	989a04da3b	pass a shared statsd instance	2021-07-05 22:46:56 -04:00
Andre Medeiros	d1c347b5d6	moar things	2021-07-05 22:18:22 -04:00
Andre Medeiros	86e6962474	fetch deps	2021-07-05 22:13:39 -04:00
Andre Medeiros	9af3586c31	some stats	2021-07-05 22:09:04 -04:00
Andre Medeiros	771e0e221e	don't spam	2021-07-05 21:55:01 -04:00