Show HN: ConvTools – generates Python code of conversions, aggregations, joins

Re:Show HN: ConvTools – generates Python code of conversions... by uryga on Sunday March 29, 2020 @04:11PM (#22722254)

sounds interesting, i'll have a look at the code when i have time. i've mostly done compiler stuff like this with the "one function with a huge switch on the expression type" approach, curious to see what the more OOP-ish way looks like.

btw: wow, that cheatsheet is exactly what i had in mind on my first comment, that's the kind of stuff i'd like in a readme! maybe a few excerpts with a link to the whole thing.

some more remarks if you're interested:

---

in that cheatsheet it'd be cool to also show the generated code for each example, maybe in a collapsible box or sth – in that context the actual semantics of a convtools expression are useful to know.

---

have you thought about some magic syntactic sugar? the current approach is kind of visually heavy, since you're basically writing an AST by hand. with some __dunder__ hacking you could easily (?) add a "magic" api like

  from convtools.magic import magic as m
  
  c.item('key') ->
  m['key']
  
  c.call_method('foo', ...) ->
  m.meth.foo(...)
  
  c.call_function('bar', ...)  ->
  m.func.bar(...)

or something similar. it might be a bit too magical for some tastes, but if you're constructing a python expression, it kind of makes sense to use python syntax for that

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by westandskif on Sunday March 29, 2020 @04:27PM (#22722400)

oh, thanks, I'll add links to cheatsheet and quickstart pages to the README, it really makes sense!

As for the magic-stuff, I was contemplating designing the API with this approach, but I changed my mind because it would be difficult to tell which python expressions are evaluated at the moment of a conversion definition AND which in the compiled code.

However if we imagine this "magic" API, then it could be even closer to normal python code:

  m["key"].some_method(...)

which would resolve everything under the hood.

===

as for the collapsable generated code examples -- I've jotted down :)

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by uryga on Sunday March 29, 2020 @05:53PM (#22723111)

> it would be difficult to tell which python expressions are evaluated at the moment of a conversion definition AND which in the compiled code

this is ofc a valid concern in any metaprogramming situation. has this been a problem in your experience? i'm guessing e.g. generating a conversion based on a list of fields is a thing someone might do, but it feels like a minority usecase (at least to me, someone with no actual experience with using the library :p)

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by westandskif on Monday March 30, 2020 @09:49AM (#22727967)

Re: whether it's been a problem in my experience -- sort of - yes

So now I'm doing my best to observe: PEP-20 the 2nd commandment with the hope that I'm not violating the 1st commandment badly :) https://www.python.org/dev/peps/pep-0020/

Also I see another upside of this no-magic syntax in that it is distinctive -- there's no way to mix up convtools-related code with any other python code.

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by carapace on Sunday March 29, 2020 @05:03PM (#22722707)

In re: the "magic", it seems to me that you could use a mock object and get the required info from it.

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by westandskif on Monday March 30, 2020 @09:40AM (#22727910)

In terms of the implementation, it's kind of trivial to implement the "magic", but it would be both confusing and inconsistent, see below.

e.g. imagine a case where you'd want to call datetime.strptime, partially initializing it at the moment of conversion definition. at the moment it is:

  c.call_func(
      datetime.strptime,
      c.item("updated"),
      "%Y-%m-%d"
  )

but it's unclear to me how would the "magic" approach deal with the case above.

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by carapace on Monday March 30, 2020 @02:04PM (#22730485)

Replace your current namespace with a mock object? You can eval or exec code with any dictionary-flavored object, and mocks can imitate dicts.

edit: no, you can't in Python 3..

I'll try to have a closer look later today.

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by germanjoey on Monday March 30, 2020 @06:02AM (#22726579)

What do you think about a DSL alternative?

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by westandskif on Monday March 30, 2020 @09:35AM (#22727867)

We would need to bring the whole python into it :/

[ Parent | on HN ]

Re:Show HN: ConvTools – generates Python code of conversions... by gigatexal on Sunday March 29, 2020 @05:19PM (#22722855)

looks really cool -- will evaluate when the API is stable -- would hate to adopt only to have it change from under me

[ Parent | on HN ]