Dependant types - Network dimension

[Cypher1]

Would it be possible to build a dependence on the dimensions of the networks so that there can't be runtime errors caused by incorrectly shaped networks?

Something like this:
https://blog.jle.im/entry/practical-dependent-types-in-haskell-1.html

Thanks

[blackgnezdo]

@Cypher1 Thanks for the link, it's interesting! You are right, it would be wonderful to have shapes be the first class citizens of type world. There are some edge cases like dynamic reshape where it may not work out, but majority of shape-related bugs can be avoided. I have a somewhat uneasy feeling about gradient construction code too.

I believe the shape-typed API can be constructed on top of the raw one that we currently have (though we may want to extend the stronger types down into generated code too). So, if you have some code lying around - do show us!

[Cypher1]

Hmm. The link I posted is my only experience with dependant types except for a few attempts at learning Agda.

Possibly when I finish my degree I'll be free to help out. I'd love to work on this.

[starper]

Here are two related blog posts: post 1 and post 2

[helq]

I'm currently working on this. I took the code from Monday Morning Haskell (the linked blog posts) and have been working on making the API less verbose, this is how far have I come so far:

main3 = runSession $ do
  (a :: Placeholder "a" '[2,2] Float) <- placeholder
  (b :: Placeholder "b" '[2,2] Float) <- placeholder
  result <- render $ a `add` b
  let (inputA :: TensorData "a" [2,2] Float) = encodeTensorData . fromJust $ fromList [1,2,3,4]
      (inputB :: TensorData "b" [2,2] Float) = encodeTensorData . fromJust $ fromList [5,6,7,8]
  runWithFeeds (feed b inputB :~~ feed a inputA :~~ NilFeedList) result

(PS: this is running code, it actually works, it type checks and fails if I change the sizes of the shapes or the names of the placeholders!. This is, I think, impressive, I've been basically just refactoring for a couple of days what James Bowen (the guy from the blog posts) wrote, I love the GHC compiler)

I hope to release the code next week. I still have some problems with some type functions (eg, Union don't work too well for this problem), but I think I could have the basics nailed down for next week.

[blackgnezdo]

@helq word of caution: you need to make sure that the original author releases their code under a compatible license if you are to contribute a derived version. Superficially, there's no license statement on the blog page which forces one to assume the most restrictive interpretation...

[helq]

@blackgnezdo I don't think this is a big problem, he released the code under the BSD license, this is the repo https://github.com/jhb563/HTensor/tree/dependent-placeholders.

Now, I'm not quite sure how to re-license that code to incorporate it to this project, this project is licensed under Apache 2. I suppose, that's not a big deal, but I'm not sure how should it be done precisely, what the LICENSE file should say? and where should the original author's name and mine's be? (Actually, I don't care much about me, I remember we concede some rights to Google once we ask the code to be merged, but it would be nice to know how to add those data to the code or to some sort of list)

[blackgnezdo]

@helq the easiest would be if the original author dual-licensed his code under Apache 2 in addition to BSD. Are you open to his @jhb563? (If you go there, changing the LICENSE file to include the name instead of "Author name" would also be a good idea)

[helq]

@blackgnezdo ok. While we wait for @jhb563, I thought I could make public the code I've been working on, https://github.com/helq/tensorflow-haskell-deptyped. After we resolve how to manage the licenses I will just copy the code to this repo and make a PR.

In the mean time, I'm working on rewriting the MNIST example to use dependent types (this could take me a week more, probably. There are many details to solve before the MNIST example works with dependent types without a hassle). I hope, after the MNIST example is working and the license thing is resolved, I'll PR.

[helq]

Ok, @jhb563 has changed the license. I should ask, is the licensing alright now? What is left to do? May I just change the license of my project too?
Thanks.

[blackgnezdo]

@helq I double checked and if we decide to depend on any external library, it has to be placed into third_party directory according to this document. Alternatively, submitting the code directly to the project via CLA process is also possible but needs to be done by the original copyright holder. Same goes for your code, you can keep it separate or sign a CLA when you submit PRs.

[helq]

@blackgnezdo I have changed the license of my code to be dual licensed into Apache and BSD3.
https://github.com/helq/tensorflow-haskell-deptyped/tree/706431224c85b2164fe815331046f6258b1afa3e

I have forked this repository and I have added my code to the root of it. I have relicensed yet again the code so it says "Copyright 2017 TensorFlow authors", is that right? What should be in the header of each license on each file, all the copyright holders? (or I ain't any copyright holder, and am I missing something?)

Anyway, here is the code https://github.com/helq/haskell/tree/deptyped [commit 91fbd8a235]

Can I make a PR now? (I have already signed the CLA) or is it necessary for @jhb563 to sign the CLA?

Thanks, and sorry for so much asking.

[blackgnezdo]

@helq apologies about delayed reply, work life interfered.

We looked a bit into the diff between the commit you sent and the tensorflow/haskell tree. My understanding is it adds a new package which doesn't require any new functionality in the tensorflow/haskell packages. If this is correct, I think we can safely keep dependent types work on the side. I'm happy to include a reference to this package to make it more discoverable for people who would like to use it.

My main motivation for this is to avoid sorting our licensing as it it seems the only copyright that can be on the code served in this repository can be that of TensorFlow Authors (so CLAs are required from all contributors).

[helq]

Wow, already a month! Sorry for the very late reply (I guess we all have work/home responsibilities).

@blackgnezdo Ok, I understand, then there is no need to merge the code (unless I start to make significant changes to this repo, I suppose). In this case, I would really appreciate if you could add a little reference to the README to indicate there are side projects extending the API.

Thank you for the replies. Oh! and happy holidays!

[fkm3]

Sure, added a link in #171. If you have a cool example or something, we can add that as a code snippet as well.