Architecture Decision Record
← Misalan rikodin shawara

Ma’auni, masu sa ido, fadakarwa

Abubuwan da ke ciki:

Taƙaitawa

Matsala

Muna son amfani da ma’auni, masu sa ido da fadakarwa, domin muna son sanin yadda aikace-aikacenmu ke aiki da kyau, da kuma lokacin da akwai matsala.

Shawara

WIP.

Matsayi

Muna tattara bayanai. Muna farawa daga ƙarshen yiwuwar bambance-bambancen: kayan aikin kyauta tsoho da aka fi ba da shawara (Nagios) da kayan aikin biya sabo da aka fi ba da shawara (New Relic).

Cikakkun bayanai

Zato

Muna son gina aikace-aikacen yanar gizo na zamani, masu sauri, masu aminci, masu amsawa, da sauransu.

Muna son saya maimakon gina.

Ƙuntatawa

Muna son kayan aiki da ke aiki da kyau da bututun devops ɗinmu da girgijen ɗora tsarinmu.

Ra’ayoyi

A halin yanzu muna bincike kan ra’ayoyi.

  • AlertManager

  • AppDynamics

  • AppOptics

  • Azure Monitor/Analytics/Insights/Dashboards

  • Bosun

  • Checkly

  • Circonus

  • Cloudwatch

  • EFK

  • ELK

  • Grafana

  • Grafana

  • Graphite

  • Graylog

  • Healthchecks.io

  • Heroku

  • icinga2

  • InfluxDB

  • Instana

  • Kafka

  • Logagent

  • Logz.io

  • Loki

  • Monitis

  • Nagios

  • Nagios

  • Nagiosgraph

  • NewRelic

  • OpsGenie

  • Outlyer

  • PagerDuty

  • Pagerduty

  • PagerDuty

  • Papertrail

  • Prometheus

  • Rollbar

  • Scalyr

  • Sematext Metrics, Logs, Experience, Tracing

  • Sensu

  • SignalFX

  • Slack

  • Splunk

  • Stackdriver

  • Stackstorm

  • Telegraf

  • Telegraf

  • Thanos

  • VictorOps

  • Wavefront

  • Zabbix

Hujja

Ya zuwa yanzu, Nagios da New Relic su ne ƙarshen bambance-bambancen. Nagios shi ne kayan aiki mafi tsufa, mafi sauƙi, kyauta, mai yiwuwa. New Relic shi ne kayan aiki mafi sabbin siffofi, mafi cikawa, na biya, mai yiwuwa. Za mu fara kimantawa da waɗannan. Idan an buƙata, za mu matsa cikin bambance-bambancen.

Ya zuwa yanzu, Zabbix yana da mafi kyawun shawarwari, kuma yana ba da cikakkun iyawa.

Ya zuwa yanzu, ELK yana da mafi kyawun shahara ta buɗaɗɗen tushe na gina-maimakon-saya.

Ya zuwa yanzu, Prometheus + Graphana suna da mafi kyawun shahara.

Tasiri

TODO.

Mai alaƙa

Shawarwari masu alaƙa

Zaɓuɓɓukan za su shafi sauƙin gwaji, telemetry, da wataƙila wasu tsarukan kamar tallafin abokan ciniki, injiniyan amincin shafi, da sauransu.

Buƙatu masu alaƙa

TODO.

Kayayyakin da aka samar masu alaƙa

TODO.

Ƙa’idoji masu alaƙa

Mai sauƙin sakewa.

Buƙatar sauri.

Bayanan kula

Kyakkyawan tarin buɗaɗɗen tushe shi ne:

  • Prometheus don ma’auni da fadakarwa bisa ma’auni

  • Grafana don nuna ma’auni

  • Elasticsearch/Logstash/Kibana (ELK) don rajistan ayyuka da abubuwan da suka faru masu tsari

  • Pushover don sanarwar wayar hannu

Saƙonnin rubutu na kyauta da saƙonnin abubuwan da suka faru masu tsari

Saƙonnin rubutu na kyauta: misali, irin abubuwan bazuwar da za ku samu a /var/log/messages, da wani abu da aikace-aikacen ya samar da gangan. Saƙonnin suna da amfani wajen gano wasu abubuwan da ke faruwa a na’urar kamar rashin ƙwaƙwalwa ko kurakuran kayan aiki, amma suna da shara da yawa.

Saƙonnin abubuwan da suka faru masu tsari: aikace-aikace ne ke samar da su, tare da ƙayyadadden tsari ko mai motsi na halaye, misali rajistan buƙatun HTTP, rajistan lissafi, shigar mai amfani.

A gaba ɗaya, yana da kyau a rubuta cikakkun bayanan kowace buƙata ta yadda za ku iya nutsewa bisa halaye. Don haka ƙara misali userid ko sessionid ga komai yana ba ku damar bin diddigi. Bin diddigi a bayyane ma yana da kyau, ba shakka. Amfani da ELK don wannan wani nau’in https://www.honeycomb.io/ ne na talaka

Graylog ya fi sauƙi

Graylog ya fi sauƙin kunnawa daga gogewata.

Prometheus yana buƙatar ɗan gyara

Gabaɗaya ina farin ciki da Prometheus don ma’auni. Fadakarwa tana buƙatar ɗan gyara, amma tana da kyau sosai. Ya dogara da aikace-aikacenku. Ina ganin ya fi kyau a fadakar bisa yanayin da mai amfani na ƙarshe ke gani, ba dalilan ƙasa ba. Misali, lokacin lodin shafi yana da kyau, adadin buƙatu a daƙiƙa ba shi da kyau. Duk da haka buƙatu sifili a daƙiƙa yana nuna cewa akwai matsala.

Fa’idar sabis ita ce suna ba da ƙarin hankali daga cikin akwatin. Gabaɗaya ina son Datadog. Sabis na iya zama masu tsada mai ban tsoro idan kuna da bayanai da yawa, kuma wani lokaci suna da tsarin farashi da ba shi da kyau ga girgije, misali caji ga kowane misali, lokacin da misalai ke da motsi. Akwai kuma bambanci tsakanin sabis inda kowace buƙata daga mai amfani ne mai biya da waɗanda ke da alaƙa da talla, don haka wani ɗan kaso kaɗan na buƙatun ne kawai ke ba ku kuɗi. Kuna iya ƙarewa da bayanai da yawa kuma ba kasafi mai yawa ba.

Ina aiki a kan wasu sabis da ke karɓar buƙatu biliyan 1 a rana, don haka yana da ma’ana a ɗauki sa ido da rajistan ayyukanmu da kanmu. Idan yawanku ya fi ƙaranci, to sabis da ake ɗauka a wajen sun fi sauƙi.

Sabis na AWS gauraye ne

Gogewata da sabis na AWS ta kasance gauraye. Sabis ɗinsu na Elasticsearch ya kasance mara dorewa, don haka muna gudanar da namu misalai don hakan. Ma’aunin CloudWatch yana da tsada, don haka gabaɗaya muna amfani da su kawai don ma’aunin matakin “kayayyakin more rayuwa” maimakon aikace-aikace, wato ma’aunin da suka shafi lafiya inda AWS zai iya sanin abin da ke faruwa fiye da manhajar da ke aiki a kan misalin. CloudWatch Logs na iya zama a hankali wajen sabuntawa kuma ba su da metadata da yawa. Gudanar da ELK yana taimakawa da hakan. Idan da gaske ina son bayanai na lokaci-na-gaske, to amfani da Kafka a matsayin hanyar jigilar rajistan ayyuka ya fi kyau. Logstash yana goyon bayan hakan sosai. Gudanar da gungun Kafka ba na masu rauni zuciya ba ne, duk da haka, akwai tsarin bututu da yawa a bayyane.

Kafka

Sharhi: Kafka na iya zama mai wahala sosai a wasu lokuta, ko Kafka na iya zama ƙarfi kamar dutse har kusan ku manta yana nan yana ɗaure komai.

Sharhi: Kafka ya kasance mai ƙarfi, amma ya ɗauki aiki mai ban mamaki kafin ya fara aiki. Ina ganinsa kamar rumbun bayanai na dangantaka amma kuna aiki ne kawai a matakin “zahiri”, misali tablespaces, fayiloli da partitions. An sami wasu lokuta da wuri inda kayan aikin gudanarwa ba su isa ba, kuma dole mu rubuta shirye-shirye don misali sake saita ƙungiyar mabukaci. http://howfuckedismydatabase.com/nosql/

Sharhi: Muna amfani da Kafka a matsayin “matsakaici” na saƙonnin rajista da wurin da za mu iya yin sarrafa kwarara a lokaci-na-gaske kan bayanan da ke zuwa daga sabobi da yawa. Idan muka sami harin DDOS, to muna buƙatar hanyar nazarin bayanai a misalai da yawa. Idan muna rubuta rajista kai tsaye daga sabobi zuwa ELK, nauyin na iya fashe gungun Elasticsearch.

Sharhi: Kafka yana da kyau a gare mu domin idan muka sami harin DDOS, to muna buƙatar hanyar nazarin bayanai a misalai da yawa. Idan muna rubuta rajista kai tsaye daga sabobi zuwa ELK, nauyin na iya fashe gungun Elasticsearch.

Sharhi: Kafka yana yin ƙarancin aiki kuma ya fi inganci, don haka zai iya sarrafa nauyin da kyau. Kuma muna jera aikin Kafka a layi kuma mu sake gwadawa. Kuma yadda Kafka ya yi nauyi ba ya shafar masu amfani da ke ƙoƙarin yin aiki mai mu’amala da Kibana, kamar yadda zai faru idan Elasticsearch yana fama.

Sharhi: Sarrafa kwarara galibi yana neman cin zarafi, misali zirga-zirga mai yawa daga IP guda a duk gungun, sannan a raba toshewar a duk gungun.

Sharhi: Plugin ɗin logstash-output-kafka yana da rashin aminci sosai a yanzu. Wasu daga cikin matsalolin da ke shafin GitHub issues ɗinsa sun cije ni, waɗanda ba su da alamar gyarawa. Ina so in daina amfani da shi, in aika kai tsaye daga aikace-aikacenmu zuwa Kafka.

Sharhi: Yanzu muna aika abubuwan da suka faru masu tsari kai tsaye daga aikace-aikacen zuwa Kafka. Babban dalili shi ne taɓa bayanan rajista sau kaɗan da guje wa karantawa da rubuta faifai sau da yawa. A tsarukan masu yawan girma, rubuta rajista na iya ɗaukar aiki fiye da aikace-aikacen kansa. Ina tunanin sa journald ya aika rajista kai tsaye ma, daga shirin C.

Loki

Ku sa ido sosai a kan Loki. Bai gama ba tukuna amma idan ya gama ina tsammanin zai fi dacewa a wannan tarin. Loki mai tara rajista ne da grafana labs suka ƙirƙira, yana amfani da irin hanyar ɗebo da tsarin tag kamar Prometheus.

Prometheus + alertmanager + Rollbar + Graylog + Grafana

Prometheus + alertmanager don ma’auni. <3 Prometheus.

Rollbar/Graylog don rajista/bayar da rahoton kuskure (akwai ɗan haɗuwa a nan; ƙaramin sabis wataƙila ba ya buƙatar duka biyun).

A halin yanzu, fadakarwa kawai tana zuwa ɗaya daga cikin ƴan tashoshin Slack da masu sha’awa suka kunna sanarwa. Idan mun fi tsanani game da kira-a-lokacin-aiki za su tafi PagerDuty/VictorOps/da sauransu.

Grafana don zane-zane da dashboards. Ina kuma jiran ganin ko sabbin siffofin rajistansu masu zuwa za su kawar da buƙatar Graylog.

Thanos

Muna amfani da Thanos a matsayin gaban saitin HA ɗinmu. Ya san yadda ake cire kwafi na ma’auratan HA.

A halin yanzu muna riƙe watanni 6 na bayanan Prometheus na gida. Wannan yana aiki da kyau sosai a gare mu. Amma ina cikin tsakiyar aiwatar da ajiyar bucket a saitin Thanos ɗinmu don ajiyar bayanai na dogon lokaci. A ka’ida, ajiyar GCS za ta kasance kusan 30% mai arha fiye da faifan dindindin na GCE na ma’auni da muke amfani da shi yanzu.

Ba ma ajiye madadin bayanan Prometheus a yanzu. Bayanan ba su da muhimmanci da gaske a gare mu fiye da samun isasshe don fadakarwa. Cikakken ɗora tarin na’urorinmu yana canzawa sosai daga shekara zuwa shekara har bayanan tarihi da suka girmi ƴan watanni ba su da ban sha’awa sosai. Yana iya zama mai ban sha’awa a samu wasu manyan alkaluma shekara-da-shekara, zan iya kafa tarin ƙa’idodin rikodin manyan alkaluma in adana su da Federation ko in bar Thanos ya kula da shi.

GYARA: Ƙaramar sanarwa, ni mai haɓaka Prometheus ne.

Prometheus HA

Ana yin HA a Prometheus ta kwafi, kuna gudanar da popers da yawa akwai hanyoyin ɗebo da yawa da cire kwafin bayanai

Ana faɗaɗawa ta hanyar yanke hanyar sadarwa da sa Prometheus daban-daban su ɗebo sassan hanyar sadarwa daban-daban

Ajiyar dogon lokaci ba ƙarfin prom ba ne amma ana ɗora shi zuwa wani abu kamar influxes ko timescaledb (wanda a fasaha yana cika akwatin HA ma) labarin da na karanta game da shi https://blog.timescale.com/prometheus-ha-postgresql-8de68d19b6f5?gi=7df160f10e07

Ban gwada abin dogon lokaci ba tukuna domin har yanzu ina gwaji ne kawai kuma ina amfani da shi don zane-zane na ɗan lokaci yayin da librenms ke sa ido kan hanyar sadarwata don dogon lokaci

Datadog + PagerDuty + Threat Stack

Muna amfani da Datadog (tare da PagerDuty) da Threat Stack kuma ba za mu iya yin farin ciki fiye da haka ba. Korafina kawai game da DD shi ne tsadar adana ma’auni.

Zabbix

Zabbix da rubutun musamman don sa ido kan kusan komai. Yana aiki kamar sihiri.

Outlyer

Ina amfani da Outlyer, amma dole ne in bayyana cewa ina aiki a nan, kuma cin abincin kai wajibi ne.

Har yanzu ana buƙatar Graylog, Sentry da Statuscake don haɓakawa.

Yana kama da son zuciya, amma bayan na yi aiki da farin ciki da Nagios da sauran tsarukan sa ido a cikin gida, zan sayi mafita da ake ɗauka a waje a kowane sabon aiki in ba da wannan wahala.

Nagios + Nagiosgraph

Muna gudanar da Nagios don dukkan sa ido da fadakarwa. Fadakarwa tana faruwa ta imel (gargaɗi da sanarwa masu mahimmanci) da sanarwar aikace-aikace masu ji (don fadakarwa masu mahimmanci).

Ana amfani da Nagiosgraph don nuni.

Wannan saiti ya yi tasiri sosai wajen sanar da mu sosai game da abin da ke faruwa a muhallinmu. Muna gudanar da sa ido kan kusan sabobi 110 masu muhimmancin manufa da kusan wuraren bayanai 760, kuma muna da wannan tsarin safiya tsawon fiye da shekaru bakwai.

Ina kuma so in tara rajista da Graylog ko ELk a wani mataki.

Prometheus + Grafana + AlertManager

Prometheus + Grafana + AlertManager ta hanyar kyakkyawar helm chart ta Prometheus Operator. Rajista har yanzu yana tafiya zuwa tsarin LogDNA birch domin mun lura ELK yana da nauyi sosai ga ƙaramin min 3 max 5 nodes ɗinmu a GKE.

DataDog + Sentry + PagerDuty.

Na kasance ina gudanar da dukkan mafitar sa ido ta kaina ta amfani da kowane irin manhaja ciki har da Nagios, Icinga, Zabbix, ELK, Greylog2, Influx, da sauran kayan aiki da yawa, amma gaskiyar ita ce akwai ƙoƙari da yawa sosai wajen gudanar da kayayyakin more rayuwar sa ido naka, musamman lokacin da za ka iya biyan wani ƙaramin kuɗi sosai don wani ya yi maka!

Biyan wasu don gudanar da kayayyakin more rayuwar sa ido yana 'yantar da abokan cinikina su mai da hankali kan gudanar da dandamalinsu maimakon sa ido kan sa ido, ma’ana ƙimar da suke samu daga dorewar dandamalinsu ta fi duk wani kuɗin Sa Ido a Matsayin Sabis nesa ba kusa ba.

Sensu + Graphite + ELK

Kamfanina yana da sha’awar abubuwan da ake ɗauka da kai sosai.

Sensu -> PagerDuty

Graphite/Grafana

ELK (Elasticsearch, Logstash, Kibana)

Prometheus + Alertmanager

Prometheus + Alertmanager don fadakarwa, tawagata tana imani cewa sa ido mai sauƙi sa ido ne mai kyau.

Sauran tsarukan kamar rajista da bin diddigi za su ba da wadataccen mahalli don tantancewa lokacin da mai kira-a-lokacin-aiki ya karɓi fadakarwa, amma ba mu taɓa gina fadakarwa a kansu ba.

Sensu + Grafana + Graylog + Kibana + NewRelic.

Sensu, grafana, graylog, kibana, newrelic.

Prometheus + Circonus

Sabis masu kayan aikin Prometheus => nazari da nuni na Circonus

icinga2 + VictorOps + NewRelic + Sentry + Slack

Muna amfani da sabis ɗin da ke ƙasa:

icinga2 don sa ido da VictorOps don fadakarwa

NewRelic don sa ido dalla-dalla kan sabis ɗin

Sentry don bin diddigin kurakurai a sabis ɗin

Slack/Imel wani ɓangare ne na fadakarwa da ke tashi daga NewRelic ko icinga2

AppDynamics + Papertrail + PagerDuty + Healthchecks.io + Stackdriver

AppDynamics

Papertrail

PagerDuty

Healthchecks.io

Stackdriver

icinga2 + elasticsearch

icinga2 tare da haɗewar elasticsearch don nazari da haɗewar graphite+grafana don zane-zane.

saboda sassaucin ƙa’idodin apply a icinga2, masu haɓakawa suna ganin sabis ɗin da suke karɓar sanarwa game da su kawai.

kuma ta icinga2 director, masu shirye-shirye za su iya ayyana nasu bincike cikin sauƙi (wanda suke yi, kowace ƴan kwanaki - bincike 100 suna fita, wasu bincike 100 suna shigowa) a babban sikeli ba tare da wata matsala ba.

DataDog + New Relic + ELK + EFK + Sentry + Alertmanager + VictorOps

Me muke da shi yanzu:

DataDog don ma’auni

New Relic don sa ido kan aikace-aikace

ELK (Elastic Search + Logstash + Kibana) don rajista

Sentry (ana ɗauka da kai) don rubuta keɓancewa

Imel + Slack + VictorOps don fadakarwa (bisa tsananin)

Me muke so mu samu:

Prometheus don ma’auni (Grafana don nuni)

New Relic (wataƙila Elastic Search APM) don sa ido kan aikace-aikace

EFK (elastic search + fluentd + kibana) don rajista. Wataƙila, Loki na Grafana zai kasance a shirye don samarwa har zuwa lokacin da muka isa nan

Sentry don keɓancewa

Alertmanager + imel + VictorOps don fadakarwa

Wavefront + Scalyr + PagerDuty + Stackstorm + Slack

Wavefront + Scalyr + PagerDuty + Stackstorm + Slack (Sanarwa: yana aiki a VMware)

Telegraf + Prometheus + InfluxDB + Grafana

Telegraf don ma’aunin sabar kamar CPU, Faifai, Ƙwaƙwalwa da Hanyar sadarwa. Muna kuma amfani da Telegraf don sa ido na SNMP kan na’urorin hanyar sadarwarmu.

Prometheus don ma’aunin aikace-aikace. Muna rubuta binciken lafiya a cikin aikace-aikacenmu da Prometheus ke ɗebowa.

InfluxDB don ajiyar jerin-lokaci. Nan ne ake aika bayanan Telegraf ɗinmu.

Grafana don dashboards da fadakarwa. Injin fadakarwar ba shi da ƙarfi sosai, amma yana yin aikin. Muna kuma harba fadakarwa zuwa Slack.

Abin da ba ni da shi a yanzu shi ne mafitar rajista ta tsakiya. ELK yana da ƙarfi amma yana da wuyar kafawa da sarrafawa, kuma ban san wani madadin kyauta da ya kusa isa a duba ba.

Sematext + Logagent + Experience

Sematext don ma’auni, don rajista, don bin diddigi, nan ba da jimawa ba don sa ido kan masu amfani na zahiri ma. Ya fi sauƙi/arha fiye da amfani da kayan aiki/sabis N daban-daban, a ganina.

Don jigilar rajista mun kasance muna amfani da rsyslog sannan muka koma Logagent.

Don bayar da rahoton faɗuwar gaba muna amfani da Sentry, amma za mu koma Experience nan ba da jimawa ba.

Sanarwa: Ni ma’aikacin Sematext ne.

Azure Monitor/Analytics + OpsGenie

Ina fata Log Analytics yana da mu’amala mafi kyau. Muna ƙaura daga splunk, wanda ya fi sauƙin kewayawa sosai.

Prometheus + Alertmanager + Grafana + Splunk + PagerDuty

Prometheus, Alertmanager, Grafana, Splunk, PagerDuty

Da gaske ba ku son gudanar da tsarin sanarwar ku. Kuna iya maye gurbin Splunk da ELK sai dai idan tawagar Tsaronku ta fi son Splunk.

Telegraf + Prometheus + Grafana + Alertmanager

Telegraf a matsayin mai tarawa, Prometheus + Alertmanager don sa ido da fadakarwa, an haɗa shi da tashoshin slack da pagerduty don fadakarwa masu mahimmanci. Grafana don nuna ma’aunin masauki.

Prometheus + Grafana + Cloudwatch + sentry + kibana + elasticsearch

Prometheus don ma’auni + fadakarwa

Grafana don dashboards na Prometheus

Cloudwatch yana sa ido kan misalan Prometheus

sentry don bin diddigin keɓancewa

kibana + elasticsearch

graylog

prometheus Push Gateway don batch/cronjobs

SOP https://github.com/rapidloop/sop don „push/forward“ ma’auni daga misali 1 na Prometheus zuwa wani

abokan ciniki ko dai suna amfani da abokan cinikin Prometheus. Muna ƙoƙarin amfani da opencensus.io a gefen abokin ciniki

PagerDuty + Monitis

PagerDuty + Monitis. Haka kuma wasu Azure Functions na musamman don gwada lafiyar wasu sabis.

Muna neman gabatar da Prometheus da Grafana a bana

Prometheus + Grafana + Bosun

Prometheus don adana bayanan jerin-lokaci. Grafana don nuni. Bosun don sarrafa fadakarwa.

Azure Monitor/Analytics/Insights/Dashboards

Shagon Azure kawai, Azure Monitor, Log Analytics, App Insights, Azure Dashboards + Pager Duty

Grafana + Monitis + OpsGenie + Slack

Grafana don sa ido kan sabis na kwantena a Kubernetes ta Prometheus

Monitis don sa ido na sabis daga ƙarshe-zuwa-ƙarshe galibi don API na yanar gizo da aikace-aikacen yanar gizo

OpsGenie don sarrafa fadakarwa

Slack don samun bayanin matsayi daga tsarukanmu

Checkly + AppOptics + Cloudwatch + Heroku + Pagerduty + Papertrail

Injiniyan (dev)ops na dogon lokaci a nan. Na girma da Nagios. Zan so in sami ra’ayoyi game da SaaS ɗina da aka kafa da kai https://checklyhq.com. Muna yin sa ido na API da sa ido na mu’amalolin shafi tare da fadakarwa mai zurfi sosai.

Na fara Checkly domin sa ido mai aiki / na roba a fannin API ya ɗan iyakance (kuma mai tsada). Sa ido na tushen burauza / na rubutu ya fi na mallaka kuma mai tsada. Muna amfani da Puppeteer kuma muna kiyaye farashi mafi ƙanƙanta gwargwadon iko.

Tarin sa idonmu:

Checkly (cin abincin kai...)

AppOptics (zane na musamman)

AWS Cloudwatch & SNS don saƙonnin SMS.

fadakarwar Heroku ta ciki.

Pagerduty

Papertrail

Instana + Logz.io + slack

Instana yana fadakar da mu a slack game da matsalolin kayayyakin more rayuwa ko raguwar aiki, kuma mun saita logz.io ya fadakar a slack kan wani adadin rajistan matakin kuskure daga matakin aikace-aikace.

SignalFX + Splunk + PagerDuty + Slack

A halin yanzu ina amfani da: SignalFX, Splunk, PagerDuty da Slack. Ba ni da sha’awar SignalFX sosai ko da yake tawagar tallafinsu tana da abokantaka da amsawa sosai. Ina son Splunk (ya cancanta idan za ku iya biya), PagerDuty da Slack.

Na kasance ina amfani da tarin TICK inda yawancin C a zahiri G ne, wato Grafana ko da yake na yi amfani da Chronograf kaɗan. Wannan ya yi kyau amma yana da wahalar sarrafawa. Rikicin gargajiya na SaaS da ɗaukar kai.

Na yi amfani da DataDog, New Relic, Graylog, ELK da BugSnag. Ina son DataDog da New Relic sosai, Graylog yana da kyau sosai. Ba ni da sha’awar ELK sosai. BugSnag yana da kyau, a zahiri ina ji bin diddigin kurakurai/keɓancewa madadin kyau ne ga cikakken sa ido kan rajista, a lokuta da yawa.

ELK + Prometheus + Grafana

Kamar sauran, muna amfani da ELK don rajista da Prometheus+Grafana don komai.

Kula da wannan saiti abu ne mai sauƙi idan kun ba kanku izinin rasa bayanai lokaci-lokaci. Misali, idan rumbun bayanan ElasticSearch ɗinmu ya shiga cikin tashin hankali (wanda abin takaici yana faruwa kowane watanni 2-3 a gare mu) ba ma damuwa da HA sai dai mu zubar da bayanan mu ci gaba da rayuwarmu. Idan dole ne kuna buƙatar HA ko riƙewa na dogon lokaci, sa’a.

Datadog + Prometheus + Grafana

Na kafa Datadog wata-da-wata domin lokacin da na zo nan, babu sa ido kuma babu fadakarwa. Ƙananan shafukanmu ne kawai ake sa ido kansu kowane minti 5 don lokacin aiki. Datadog babu shakka shi ne mafi sauƙin kafawa. Idan na gama magance sauran matsalolin, zan koma Prometheus+Grafana. Ban yanke 100% ba tukuna kan sarrafa rajista.

Nagios + ELK

Muna da kayayyaki fiye da 100+ da muke tallafawa.

Don cikin-gida, galibi Nagios da ELK. Don girgije, muna komawa daga DataDog zuwa NewRelic.

Datadog vs. Site24x7 + StatusCake + PagerDuty + SumoLogic + Slack

Mun kasance muna amfani da datadog amma mun ga yana da tsada sosai ga buƙatunmu. Kada ku fahimce ni ba daidai ba yana da ban mamaki, amma yana da babban kuɗi. Mun iya kafa site24x7.com da biyan kuɗin shekara na kusan watanni 2-3 na kuɗin DD.

Tarin sa idonmu:

Site24x7 - APM, sa ido kan URL na waje, sa ido kan kwararar mail na SMTP, ƙarewar ssl da sa ido kan tsari.

StatusCake - don sa ido kan URL da tabbatarwa - Shi ne madadinmu idan site24x7 ya rasa wani abu (ba ya rasawa) amma SC ya fi sassauci don sa ido kan tashar waje da sabis don buƙatunmu.

Kayan aikin biyu suna ɗaga zuwa PagerDuty, sannan muna samun ɗagawarmu a slack.

SumoLogic - don sa ido kan rajista (kayan aiki ne mai kyau amma ɗan sarƙaƙiya ga buƙatunmu)

Daga slack za mu iya ack, ko gyara fadakarwar.

Sannan muna da sarrafa abubuwa da yawa ta atomatik na site24x7 da suke haɗawa da commando.io don ‘BedOps’ kamar yadda muke kira - inda fadakarwa ke tashi, muna fara wasu rubutu ko sarrafa ta atomatik a matsayin ƙoƙarin gyara lamarin (kashi 99% na lokaci sarrafa ta atomatik + rubutunmu suna kiyaye mu daga matsala)

Muna da littattafan aiki na ciki a KB ɗinmu don lokacin da sarrafa ta atomatik ta gaza ko idan akwai wani abu a wajen iyaka da ke buƙatar gyarawa.

Prometheus + AlertManager + Grafana + Stackdriver

Prometheus (Operator) / AlertManager / Grafana don ma’auni a gungunmu na GKE da VMs.

Google Stackdriver don Rajista (domin an haɗa shi kuma yana aiki ta tsohuwa kuma a halin yanzu ya isa ga buƙatunmu).

Zabbix

Zabbix ga komai. Babu ƙarin manhaja da ake buƙata.