INT 21h

Hi, I am Vladimir Smagin, SysAdmin, DevOps and barely good guy. Telegram Email / GIT / Микроблог / Thingiverse / RSS / GPG

Warm Image operator for Kubernetes

№ 11010 В разделе "Sysadmin" от May 28th, 2020,
В подшивках: , , ,

For example, you have huge image with your software and running POD on node. When POD moving to another node your image downloads to new node minute or two. This operator forces nodes to download image before rescheduling, so POD starts faster.

It runs /bin/sh with infinite loop on specified image as DaemonSet with additional options like NodeSelector, Affinity or resource limits. You can specify custom command if your image not contains /bin/sh interpreter or you want to run own script.

Your first warmer:

apiVersion: blindage.org/v1alpha1
kind: WarmImage
metadata:
  name: mongo4
spec:
  image: mongo
  version: "4"
  nodeSelector:
    node-role.kubernetes.io/master: ""

Now you warmed mongo:4 on all master nodes.

Repository here https://git.blindage.org/21h/warm-image-operator

Нет комментариев »

История провала: webdav и nginx для Windows

№ 10985. В разделе " Sysadmin " от April 25th, 2020

В подшивках: , ,

How to move snap application between computers with snapshots

№ 10960. В разделе " Sysadmin " от March 29th, 2020

В подшивках: ,

wal-g and Postgresql

№ 10938 В разделе "Sysadmin" от March 20th, 2020,
В подшивках: , ,

Initials

Master database: postgresql-10, in production
Backup storage: minio with empty pg-walg-backup bucket

Task: Create slave postgresql-10 server with recovery

Preparing for both servers

Install wal-g binary manually or use my Ubuntu repository (install walg-lzo).

Create special script for wal-g with variables, save to /var/lib/postgresql/walg-postgresql

#!/bin/bash
source /etc/profile

export AWS_ENDPOINT=http://minio.myserv.com:9000
export AWS_ACCESS_KEY_ID="KEYNAMEKEYNAMEKEYNAME"
export AWS_SECRET_ACCESS_KEY="SECRETSECRETSECRETSECRETSECRETSECRET"
export WALE_S3_PREFIX="s3://pg-walg-backup/"
export AWS_S3_FORCE_PATH_STYLE="true"
export AWS_REGION=us-east-1
export PGHOST=/var/run/postgresql
export PGUSER=postgres
export PGPORT=5433

wal-g $@

It is not secure to store this file global readable. Best choice to place it in postgres home directory without read by others:

chmod 500 /var/lib/postgresql/walg-postgresql
chown postgres:postgres /var/lib/postgresql/walg-postgresql

Master server

Add to postgresql.conf:

archive_mode = on
archive_command = '/var/lib/postgresql/walg-postgresql wal-push %p'

Ok, now run backup-push to create full backup:

/var/lib/postgresql/walg-postgresql backup-push /pg_data/10/main

Slave server

Stop postgresql and remove data directory, wal-g requires empty directory before restore.

Now run:

/var/lib/postgresql/walg-postgresql backup-fetch /pgdata/10/main LATEST

Create /pgdata/10/main/recovery.conf:

standby_mode = 'on'
restore_command = 'walg-postgresql wal-fetch "%f" "%p"'

Now see logfile for wal restoring, database in recovery mode.

Нет комментариев »

Postgresql logical backup failed, pg_dump said: PQgetResult() failed

№ 10947 В разделе "Sysadmin" от March 20th, 2020,
В подшивках: , ,

You made streaming backups and restoring with walg and now trying to make logical backups.

But some shit ruined your plans:

pg_dump: Dumping the contents of table "<table>" failed: PQgetResult() failed.

Many sites suggests to enable hot_standby_feedback in postgresql.conf, but actually you need also increase max_standby_archive_delay and max_standby_streaming_delay. I set 1 hour.

Нет комментариев »

Яндекс.Метрика

Fortune cookie: There was a young lady of Dee Who went down to the river to pee. A man in a punt Put his hand on her cunt, And God! how I wish it were me.