News for hackers. Stuff that matters.
Re:Goodbye Python, Hello Go by arendtio on Friday August 03, 2018 @06:15AM (#17678615)
Does someone know a ressource where I can easily learn how to switch from Bash to Go?

To give some context: When it comes to serious programs I use Go already, but often I just want to code a little script to watch a website for changes for example:

  #!/usr/bin/env bash
  url="https://example.com"
  mem="memory.txt"
  oldsum="$(cat "$mem")"
  newsum="$(curl -sL "$url" | sha512sum)"

  if [ "$oldsum" != "$newsum" ]; then
    xmpp_notify "$url changed"
    echo "$newsum" > "$mem"
  fi
I haven't tried it yet, but in my mind it feels like writing such a program would be more time consuming/complex in Go than writing it in bash. Neverthelss, I feel like Go is overall a much better language than Bash (with all those quotes), so I would like to transition to writing more short programs in Go and for that I would like to learn how to do things like reading/writing/grepping files as easily in Go as I do it in Bash. Any idea if someone wrote some hints together already?
Re:Goodbye Python, Hello Go by chomp on Friday August 03, 2018 @06:22AM (#17678643)
The benefit of bash is that it has system binaries as first class citizens; you can leverage the hundreds of thousands of lines of GNU projects with a single keyword. It's hard to beat that expressiveness.

Saying "Go is overall a much better language" is like saying a wrench is a better tool than a screwdriver. You should keep both in your toolbelt and switch out when the need arises.

For your question, start with the regexp and os package, learn how to open a file, read in the contents, and then while iterating over the contents, apply a regex to the text. That should get you to the point that you can search plaintext for character strings. Good luck.

Re:Goodbye Python, Hello Go by IshKebab on Friday August 03, 2018 @07:47AM (#17679026)
Writing a program like that would not be too difficult in Go. It's something like this:

    func main() {
      url := "https://example.com"
      mem := "memory.txt"
      oldsum, err := ioutil.ReadFile(mem)
      if err != nil {
        log.Fatal(err)
      }
      
      // Download URL
      resp, err := http.Get(url)
      if err != nil {
        log.Fatal(err)
      }
      defer resp.Body.Close()
      body, err := ioutil.ReadAll(resp.Body)
      if err != nil {
        log.Fatal(err)
      }
      
      // Hash it
      newsum := sha512.Sum512(body)
      
      if oldsum != newsum {
        err = exec.Command("xmpp_notify", url + " changed").Run()
        if err != nil {
          log.Warn(err)
        }
        err = ioutil.Write(mem, newsum)
        if err != nil {
          log.Error(err)
        }
      }
    }
Don't listen to the naysayers. Go is fantastic for scripts like this - it's really good at network stuff, has a ton of built in functions, etc. And you get actual robustness and error checking that isn't a complete joke. Look how many ways your Bash script could fail! That's fine if you're literally watching the output with your eyes. If not it's going to cause issues when `curl` fails for example.

The only downside really is that you have to compile the Go code, which is fast but not as fast as just interpreting a Bash script (the first time).

Another possible issue is executable size - if you have a lot of Go scripts they may take up a fair bit of space.

I wish Go had an interpreter / JIT mode for scripts.

Edit: Before someone says "but that's 3 times longer than the bash code!" please write the equivalent Bash code that has proper error checking.

Re:Goodbye Python, Hello Go by boris on Friday August 03, 2018 @08:07AM (#17679144)
Just add this as a second line?

  trap "exit 1" ERR
Re:Goodbye Python, Hello Go by majewsky on Sunday August 05, 2018 @03:57PM (#17693178)
I think you mean:

  set -euo pipefail
Re:Goodbye Python, Hello Go by arendtio on Friday August 03, 2018 @09:30AM (#17679640)
Thanks for giving an equivalent example in Go :-)

I think for this small size of an example program the error checking in Go is pure overhead, but as soon as the program starts to have a few more functions it will be very useful to have more options than just to abort on fail (Bash trap style / set -e). So I don't have any problem with 'polluting' my source with `if err != nil` checks ;-)

What I am more concerned about are the different ways I have to call functions from different libraries like

  resp, err := http.Get(url)
  ...
  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
That is a lot more 'special' than the other function calls and the bash version and probably requires more documentation reading than '$ curl --help'. But hey, thanks to your example I have a starting point and probably are going to experiment with it in the new future.

For all the others who wrote that Bash is the right tool for the job: I read your warnings, opinions and suggestions and will keep them in mind while exploring the possibilities of using Go for script like jobs.

Re:Goodbye Python, Hello Go by h1d on Friday August 03, 2018 @08:34AM (#17679297)
That is so repetitive to handle errors. Doesn't go have a cleaner way of handling errors than making the code look far less readable without unified error handling like exceptions?
Re:Goodbye Python, Hello Go by IshKebab on Saturday August 04, 2018 @05:07PM (#17688434)
It is a little repetitive. The best solution is Rust's Result system. But I'd say Ho's error handling system is second to that.

You might be thinking "but... exceptions!". They are a pretty bad solution. The problems are:

* Error handling is hidden - it's impossible in most implementations to know which exceptions can be thrown by a function * Flow control context is lost. If you wrap lots of lines with try {} you don't know which one caused the error and can't add context information. You end up with shit error messages like "file not found" rather than "error opening config file foo.cfg: file not found". The solution is to wrap every line in a separate try/catch but that is just Go-style error handling but shitter and even more verbose.

Go is a good balance. If you want proper error handling you have to actually write it. Yes it is a little tedious but that's what it takes to do it properly.

If you don't want to do it properly then fine, use Bash and set -e. But accept that it will be shit.

Re:Goodbye Python, Hello Go by ainar-g on Friday August 03, 2018 @06:34AM (#17678690)
As a professional gopher, don't.

Just a few months ago I replaced an almost 100-line Go "script" with two lines in the Makefile. The fact that Go is great for big projects doesn't automatically make it good for everything. Go is for programs, not scripts.

Re:Goodbye Python, Hello Go by arendtio on Friday August 03, 2018 @09:41AM (#17679725)
Thanks for your warning :-)

My problems come from the other end: My Bash scripts tend to become larger. A few weeks ago I wrote a 'cache' utility function (very useful to cache curl requests for example). A few days ago I wanted to use it in a second script was just about to write a package manager for bash modules... I think that is the kind of situation you are getting into when the line count of a bash script increases and you are trying to do proper software development ;-)

So I am not going to build every script in Go from today on, but I would like to learn a bit more on when Go is a viable alternative to Bash.

Re:Goodbye Python, Hello Go by watshn on Sunday August 05, 2018 @07:56PM (#17694240)
Being such a concise language, Haskell could be a good option here. You can do quick scripting while still having a powerful type system (especially compared to Go's) backing you up. I somewhat quickly typed this code up and haven't tested it, but it should replicate that bash script pretty closely:

  #!/usr/bin/env stack
  -- stack script --resolver=lts-12.2
  
  import System.Process (system)
  
  import Control.Lens
  import Crypto.Hash.SHA256
  import qualified Data.Bytestring.Lazy as B
  import Network.Wreq
  
  url = "https://example.com"
  mem = "memory.txt"
  
  main = do
      oldsum <- B.readFile mem
      newsum <- (hashLazy . view responseBody) <$> get url
      when (oldsum /== newsum) $ do
          exitCode <- system $ "xmpp_notify " ++ url ++ " changed"
          B.writeFile mem newsum
I didn't make use of it, but there's also the Turtle library which aims to provide a fairly solid shell scripting experience, providing a lot of coreutils as simple Haskell functions. You also get a REPL in the form of GHCI, which comes default in Haskell installations.
Re:Goodbye Python, Hello Go by dilap on Friday August 03, 2018 @10:27AM (#17680101)
I love Go and use it for allllmost everything, but I think bash is still better for something like this. The ease with which it can run programs and capture output is the killer feature.

I'd love to see some future bash like language that's strongly typed, has a consistent and simple syntax, but retains these advantages.

Re:Goodbye Python, Hello Go by baq on Friday August 03, 2018 @06:34AM (#17678688)
i'd say bash is the perfect tool for the problem that you demonstrated. go may be a better language but it's just another tool.
Re:Goodbye Python, Hello Go by IshKebab on Friday August 03, 2018 @07:34AM (#17678958)
Depends how much you care about robustness. If the answer is "more than zero" than Bash is a terrible tool.
Re:Goodbye Python, Hello Go by rezeroed on Friday August 03, 2018 @06:23AM (#17678645)
Go is the wrong tool for that. I'm no fan of bash, but it's often the the quick, clear (enough) solution for small tasks. I'm not a python dev - maybe that would be a suitable compromise?
Re:Goodbye Python, Hello Go by lalaithion on Friday August 03, 2018 @01:18PM (#17681635)
for me, I use bash for anything less than 10 lines long, python for anything between 10 and 500 lines, and then I switch to a heavier programming language for anything longer.
Re:Goodbye Python, Hello Go by swaroop on Friday August 03, 2018 @07:42AM (#17678995)
Or you could use Python with https://pypi.org/project/plumbum/ :)