Saturday, September 16, 2017

Slice chunking in Go

Slice chunking in Go

package main

import "fmt"

var (
 logs   = []string{"a", "b", "c", "d", "e", "f", "g", "h", "i", "j"}
 numCPU = 3
)

func main() {

 var divided [][]string

 chunkSize := (len(logs) + numCPU - 1) / numCPU

 for i := 0; i < len(logs); i += chunkSize {
  end := i + chunkSize

  if end > len(logs) {
   end = len(logs)
  }

  divided = append(divided, logs[i:end])
 }

 fmt.Printf("%#v\n", divided)
}

Reference:

https://stackoverflow.com/questions/35179656/slice-chunking-in-go

Saturday, September 9, 2017

Why is a Goroutine’s stack infinite high cpu spikes:

Why is a Goroutine’s stack infinite high cpu spikes:

Empty loop:

for{
}

uses 100% of a CPU Core.

The proper way to wait for some operation depending to the use case you may use:

- sync.WaitGroup like this
- select {}
- channels
- time.Sleep
- time.After

One of the key features of Goroutines is their cost; they are cheap to create in terms of initial memory footprint (as opposed to the 1 to 8 megabytes with a traditional POSIX thread) and their stack grows and shrinks as necessary. This allows a Goroutine to start with a single 4096 byte stack which grows and shrinks as needed without the risk of ever running out.
There is however one detail I have withheld until now, which links the accidental use of a recursive function to a serious case of memory exhaustion for your operating system, and that is, when new stack pages are needed, they are allocated from the heap.
As your infinite function continues to call itself, new stack pages are allocated from the heap, permitting the function to continue to call itself over and over again. Fairly quickly the size of the heap will exceed the amount of free physical memory in your machine, at which point swapping will soon make your machine unusable.
The size of the heap available to Go programs depends on a lot of things, including the architecture of your CPU and your operating system, but it generally represents an amount of memory that exceeds the physical memory of your machine, so your machine is likely to swap heavily before your program ever exhausts its heap.

Reference:

https://stackoverflow.com/questions/39493692/difference-between-the-main-goroutine-and-spawned-goroutines-of-a-go-program

http://dave.cheney.net/2013/06/02/why-is-a-goroutines-stack-infinite

Trouble reading packets from a socket in Go

Trouble reading packets from a socket in Go

buf needs to have a definite size. 0-length slice won't work.

Declare it as:

var buf = make([]byte, 1024)

func handleClient(conn net.Conn) {
        defer conn.Close()

        var buf [512]byte

        for {
                n, err := conn.Read(buf[0:])

                if err != nil {
                        fmt.Printf("%v\n", err.Error())
                        return
                }

                fmt.Printf("Got: %d, %s\n", n, buf)

                str := strings.Repeat("A", 16000000)
                _, err2 := conn.Write([]byte(str))

                if err2 != nil {
                        fmt.Printf("%v\n", err.Error())
                        return
                }
        }
}

Reference:

https://stackoverflow.com/questions/2270670/trouble-reading-from-a-socket-in-go

Why can not I copy a slice with copy in golang?

Why can not I copy a slice with copy in golang?

The builtin copy(dst, src) copies min(len(dst), len(src)) elements.

Copy returns the number of elements copied, which will be the minimum of len(src) and len(dst).

So if your dst is empty (len(dst) == 0), nothing will be copied.

Try:

tmp := make([]int, len(arr))

Reference:

https://stackoverflow.com/questions/30182538/why-can-not-i-copy-a-slice-with-copy-in-golang

https://golang.org/ref/spec#Appending_and_copying_slices

docker exec is not working in cron

docker exec is not working in cron

The docker exec command says it needs "pseudo terminal and runs in interactive mode" (-it flags) while cron doesn't attach to any TTYs.

Try using the -d flag instead of -it:

# /bin/sh -c "/usr/bin/docker exec -u root -d exp_mongo_1 /bin/sh /backup.sh"

Reference:

https://stackoverflow.com/questions/37089033/docker-exec-is-not-working-in-cron

Statically compiled Go programs, always, even with cgo, using musl

Statically compiled Go programs, always, even with cgo, using musl

Reference:

https://dominik.honnef.co/posts/2015/06/go-musl/