- line 53-56 (in your code), which correspond to line 41-45 in Snowkylin's code
- line 70 vs 69, where you've used a build in function instead of the explicit compositional form of softplus
- finally the major change I can see are various variable initialisation schemes 147-178 in your code, which is handled
by line 157-185 in snowlkylin's code
and some miscellaneous places where you've used a NamedTuple instead of a dictionary. And yes the choice of initialisation is a valid improvement. As far as I can tell Snowkylin's implementation is not broken, but just might learn more slowly because of a different state initialisation scheme. It does not suffer the NaN problem for example.
It should be communicated very clearly also in the paper that the code you are releasing is derived from another open-source implementation, with a precise explanation of the changes you have made. Also you can't change the LICENSE and copyright notice of open-source code to yourself without permission. I believe even in the current form your Github repo is in breach of the LGPL3. For more details how to use the LGPL properly, please refer to https://www.gnu.org/licenses/gpl-howto.en.html. Notice that the original repository actually does not carry out all those steps properly.
Namely it is missing LICENSE and Copyright notices in every file. It also doesn't have an explicit copyright notice anywhere.
Re:Implementing Neural Turing Machines
by andreyk on Thursday July 26, 2018 @01:04PM
(#17618987)
Indeed, if these are so similar your repo really ought to be a fork of the prior repo, and you should certainly acknowledge them more in the paper (make clearer what your contributions are).
- line 53-56 (in your code), which correspond to line 41-45 in Snowkylin's code
- line 70 vs 69, where you've used a build in function instead of the explicit compositional form of softplus
- finally the major change I can see are various variable initialisation schemes 147-178 in your code, which is handled by line 157-185 in snowlkylin's code
and some miscellaneous places where you've used a NamedTuple instead of a dictionary. And yes the choice of initialisation is a valid improvement. As far as I can tell Snowkylin's implementation is not broken, but just might learn more slowly because of a different state initialisation scheme. It does not suffer the NaN problem for example.
It should be communicated very clearly also in the paper that the code you are releasing is derived from another open-source implementation, with a precise explanation of the changes you have made. Also you can't change the LICENSE and copyright notice of open-source code to yourself without permission. I believe even in the current form your Github repo is in breach of the LGPL3. For more details how to use the LGPL properly, please refer to https://www.gnu.org/licenses/gpl-howto.en.html. Notice that the original repository actually does not carry out all those steps properly.
Namely it is missing LICENSE and Copyright notices in every file. It also doesn't have an explicit copyright notice anywhere.