News for hackers. Stuff that matters.

manthideaal

How not to write a book about Keras

I am reading a book about keras and the author introduces new deep learning techniques and choose the problem of estimating the temperature from local data to compare performance of those techniques. The baseline is predicting tomorrow temperature by today temperature and no wonder deep learning cannot do any better. So how are you going to compare deep learning methods solving a problem that...

( Read More... | no comments | 3 points | on HN )
Re:Show HN: 33 line React on Saturday April 04, 2020 @08:05AM (#22777784)
to merge the properties of two objects you could use

  let merged = {...obj1, ...obj2};  See (1) which has 2808 votes.
(1) https://stackoverflow.com/questions/171251/how-can-i-merge-p...
Re:Show HN: 33 line React on Saturday April 04, 2020 @06:29AM (#22777429)
In the source code to perform el.classList = v.classes the author uses:

  for (const name of v.classes)
    if (!el.classList.contains(name)) el.classList.add(name)
  for (const name of el.classList) 
    if (!v.classes.includes(name)) el.classList.remove(name)
Why the standard decided to make classList read-only?

Could ele.className=(v.classes).join(" ") be a valid and performant solution?, perhaps is to avoid the string to token traslation for performance reasons?, then why don't they include a classList.set method?

Re:Illustrated FixMatch for semi-supervised learning on Friday April 03, 2020 @02:12PM (#22772356)
I wonder if a two step process could work better than this, first use a variational autoencoder or simple an autoencoder then use it to train the labeled sampled.

In (1) there is a full example of using the two step strategy but using more labeled data to obtain 92% of accuracy. Someone can try changing the second part to use only ten labels for the classifying part and share results?

(1) https://www.datacamp.com/community/tutorials/autoencoder-cla...

Edited: I found a deep analysis in (2), in short for CIFAR 10 the VAE semi-supervised learning approach provides poor results, but the author has not used augmentation!

(2) http://bjlkeng.github.io/posts/semi-supervised-learning-with...

Re:Mediasoup – WebRTC Video Conferencing on Friday April 03, 2020 @10:25AM (#22769419)
Thanks for all the info. I think that mediasoup is a very good SFU. I wish you the best and I hope mediasoup SFU to become a crucial tool for rtc.

Edited: In a recent article (1) it seems that multicast is better than SFU in webrtc. In the PhD. Thesis (2) a hybrid model is used: Hybrid multicast-unicast video streaming over heterogenous cellular networks.

(1)https://ieeexplore.ieee.org/document/8811590 (2) https://summit.sfu.ca/item/16802

Re:Mediasoup – WebRTC Video Conferencing on Friday April 03, 2020 @09:22AM (#22768876)
Each receiver sending feedback does not prevent the server for using datagrams with multiple destinations. I can see that each peer use a different resolution and bitrate but that is another layer, is like sending information at several resolutions and each peer selecting the best one.

Edited: I must learn something about multicast in IPv6, the idea seems interesting.

Re:Mediasoup – WebRTC Video Conferencing on Friday April 03, 2020 @08:53AM (#22768683)
IPv6 could have and address for "reachable by default", I find that very useful, also IPv6 allow many addresses so that is not wasteful.
Re:Mediasoup – WebRTC Video Conferencing on Friday April 03, 2020 @08:51AM (#22768664)
In order to send a datagram to multiple IP the first, and naive idea, one can think of is to change the datagram protocol to allow for multiple destination. Today, 2020, is the right time. I am thinking about platforms that have hundred or thousand of simultaneous receiving ends, so that the branching point occurs near the destination. Again, googling this proposal is not new (1) RFC 1770, category informational.

Edited: It seem that RtcDataChannel can be used with SFU, example LiveSwitch in 2018, but they don't use multiple destination datagrams (3)

More on similar proposal (2).

(1) IPv4 Option for Sender Directed Multi-Destination Delivery. The Selective Directed Broadcast Mode (SDBM) is an integral part of the U.S. Army standard for tactical data communication networks as defined in MIL-STD-188-220().

(2) https://www.researchgate.net/publication/238663190_IPv4_Opti...

(3) https://www.frozenmountain.com/developers/blog/archive/indus...

Re:Mediasoup – WebRTC Video Conferencing on Friday April 03, 2020 @06:04AM (#22767724)
To understand the terms: webrtc, stun, turn, mesh, sfu, mcu, ice and trickle ice, there is (1). 15 minutes to understand what is all this about. What about IPv6 stun and turn?, it seems other people asked the same idea I thought: (2) bout all of this, one is the answer is: As IPv6 takes over the complexity of new networks, STUN and ICE will become irrelevant. I think that with the surge in video conferences and rtc, ipv6 with take off.

In my very humble opinion, I would suggest to reserve some address space in IPv6 for rtc, so that a peer is able to adopt a new special ip reserved for rtc. Nothing new under the sun, in 2014 someone comment along this line of thought (2) and (3).

So what are we waiting for?

(1) https://webrtcglossary.com/ (2) https://www.quora.com/Will-the-IPv6-result-in-the-death-of-S...

(3) 2014, AshleysBrain, https://news.ycombinator.com/item?id=7496986 I think the solution is IPv6. Once every device on the Internet is uniquely addressable again, we can do away with these NAT hacks and two endpoints should be able to reliably connect to each other again, no matter where they are. Of course, that's assuming we don't get more short-sighted engineering that breaks things again...

Re:ACM Prize in Computing Awarded to AlphaGo Developer on Thursday April 02, 2020 @05:13AM (#22757347)
Another link with pdf with lectures, exam and assigment and a link to video of lectures in (1)

(1) https://www.davidsilver.uk/teaching/

Re:When to assume neural networks can solve a problem on Monday March 30, 2020 @03:33PM (#22731433)
You are right. Also the Universal Approximation theorem (1) for neural networks guarantees that neural networks can approximate continuous function on compact subsets of R^n, in this case max(x,y).

argmax([x,y]) = (sign(x[0]-x[1])+1)/2

Going beyond continuous functions, can deep learning be used for primality test?

(1) https://en.wikipedia.org/wiki/Universal_approximation_theore...

Re:When to assume neural networks can solve a problem on Monday March 30, 2020 @01:34PM (#22730125)
it seems that neural networks have problem for computing the maximum function, and a human can compute the maximum easily, so it seems that the three heuristic rules don't work in this case.

(1) https://datascience.stackexchange.com/questions/56676/can-ma...

IANAL, but if the purpose of taking screenshot every 30 minutes is to control the work of the employee you must know that in the EU you have the right to be informed about any measure taken to control you.

If you can convince the judge that taking the screenshot has other purpose then GDPR doesn't apply.

From (2): The WP29 outlines that a DPIA is likely to be required if «a company systematically monitor(s) its employees’ activities, including the monitoring of the employees’ work station, internet activity» since it implies a «systematic monitoring and data concerning vulnerable data subjects» (23), form GDPR and Personal Data Protection in the Employment Context CLAUDIA OGRISEG

In (1) at point 8: the employeer has to inform the employee about: (i) whether and when monitoring is applied. (ii) the purpose of data processing, (iii) the means used for data processing.

https://legalict.com/factsheets/privacy-monitoring-work-gdpr...

Point 2) What king of personal data does an employer process, includes: Remote management of all mobile devices, such as phones and laptops;

Using so many microservices seems like a good use of erlang, googling I found: (1)

(1) http://blog.plataformatec.com.br/2019/10/kubernetes-and-the-...

Re:Real Not Complex: Free Math Resources on Saturday March 28, 2020 @04:20PM (#22714036)
A more recent version of the last link in the Statistics and machine learning section is (1)

(1) https://github.com/percyliang/cs229t/blob/master/lectures/no...

Some more info from (1): The new Netwalker phishing campaign is using an attachment named "CORONAVIRUS_COVID-19.vbs". Related: Also from (2): APT36 uses two lure formats in this campaign: Excel documents with embedded malicious macros and RTF documents files designed to exploit the CVE-2017-0199 Microsoft Office/WordPad remote code execution vulnerability.

from (3) .Ransomware Gangs to Stop Attacking Health Orgs During Pandemic

(1) https://www.bleepingcomputer.com/news/security/netwalker-ran...

(2) https://www.bleepingcomputer.com/news/security/nation-backed...

(3) https://www.bleepingcomputer.com/news/security/ransomware-ga...

TP + FN means true positives plus false negatives, that is the number of deseased people. P should be the number of people that gives positive in the test, since TP means true positive in that sense.

Also TP + FN = Prevalence of the desease, but using P for prevalence and for positive creates a lot of confusion in a wikipedia page to explain a concept.

My plea is that I don't want people to get confused. On top of it there is a warning on that page of wikipedia: This article may be too technical for most readers to understand. I couldn't understand why they used that notation. So I ask to delete that page because is going to confuse people.

I can't understand why one would use P in such a way that P = TP + FN, this is going to be a real mess.

Afortunately there is the following page in which they only use TP,FP, TN, FN that is true/false and positive/negative. So replace that page with https://en.wikipedia.org/wiki/Precision_and_recall or change notation. Please.

P = TP + FN is something that I would never put on paper.

I think that assigning credit to individual features doesn't work if there is a strong interaction among features. I would like to see a method of constructing a map that describes sets of features with strong interaction, that is new features that are not reducible to individual features in order to explain how the model work.

Edited, (1) is a good first step at shapely value theory and applications:

(1) https://medium.com/datalab-log/understanding-the-impact-of-f...

Related: Poor man integrator (1), a 99 lines of maple symbolic integrator that uses an heuristic (parallel integrator).

http://www-sop.inria.fr/cafe/Manuel.Bronstein/pmint/index.ht...

Clearly, if your generator for equations is limited the system can learn the dictionary lhs (integral) => rhs (derivative) to go rhs => lhs. (integrate)

if AutoML-Zero is going to be more than a grid-like method then I think it should try to learn a probabilistic distribution over (method, problem, efficiency) and use it to discover features for problems using an auto-encoder in which the loss function is a metric over the (method,efficiency) space. That means using transfer-learning from related problems in which the similarity of problems is based of the (method,efficiency) differency.

Problem P1 is locally similar to P2 if (method,efficiency,P1) meassured in computation time is similar to (method,efficiency,P2) for method in a local space of methods. The method should learn to classify both problem and methods, that's similar to learning words and context words in NLP or matrix factorization in recommendation systems. To sample the (space,method,efficiency) space one need huge resources.

Added: To compare a pair of (method,problem) some stardardization should be used, for linear problems related to solving linear systems the condition number of the coefficiency matrix should be used as a feature for standardization and, for example in SAT an heuristic using the number of clauses and variables should be used for estimating the complexity and normalization of problems. So the preprocessing step should use the best known heuristic for solving the problem and estimating its complexity as both a feature and a method for normalization. Heuristic and DL for TSP is approaching SOTA (but concord is better yet).

Finally perhaps some encoding about how the heuristic was obtained could be used as a feature of the problem (heuristic from minimum spanning tree, branch and bound, dynamic programming, recurrence, memoization, hill climbing, ...) as an enumerative type.

So some problems for preprocessing are: 1) What is a good heuristic for solving this problem. 2) What is a good heuristic for bounding or estimating its complexity. 3) How can you use those heuristics to standardize or normalize its complexity. 4) How big should be the problem so that the assymptotic complexity takes over the noise of small problems. 5) How do you encode the different types of heuristics. 6) How do you value the sequential versus parallel method for solving the problem.

Finally, I wonder if once a problem is autoencoded then if some kind of curvature could be defined, that curvature should be related to the average complexity of a local space of problems, also transitions like in graph problems should be feautured. The idea is using gems of features to allow the system to combine those or discover new better features. Curvature could be used for clustering problem that is for classification of types of problems. For example all preprocessed problems for solving a linear system should be normalize to have similar efficiency when using the family F of learning methods otherwise a feature is introduced for further normalization. For example some problems could require to estimate the number of local extrema and the flat (zero curvature extend of those zones)

With all due respect to Minsky, I find this zen style story a little silly. If Minsky want to say something informative why don't he use formal concepts like Jeffreys priors, mixing time, high dimensional varieties, minimun description length, entropy, etc. Is that style of telling stories a projection from a high dimensional mind to a zero dimensional dumb style space?, is that a PCA reduction from ideas to cliches? I apologize in advance from being harsh, but I am entitle to speak from my heart and I reiterate my appreciation for Minsky's work.

It should be nice using a more informative language for giving advice. If this story is tagged as "popular story for dummies" I would feel we are making real progress.

Just one of Minsky great ideas related to reinforcement learning: The credit assignment problem:How do you distribute credit for success among the many decisions that may have been involved in producing it?, in "Steps Toward Artificial Intelligence" (Minsky, 1961): All of the methods we discuss in this book are, in a sense, directed toward solving this problem.

That book is linked from HN and it has just one comment, so I think that NDNS, no dumb nerd stories, will never become popular.

(1) https://news.ycombinator.com/item?id=10972522

More from (2) Minsky in 1951 built the world's first “randomly wired neural network learning machine,” called the stochastic neural-analog reinforcement computer (snarc)

https://www.geek.com/blurb/marvin-minsky-ai-has-been-brain-d...

A fair paper: Exploring Randomly Wired Neural Network for Image Recognition.

Was Sussan at the edge of envisioning deep learning?, then in fact the room has dissapeared!

The idea of selecting the 25 features based on maximum correlation seems to be weak because it should introduce a lot of collinearity. In chapter 6 of the ISLR book there are many methods to work in high dimension, that is when number of features is bigger than number of samples. For example principal components regression, partial least squares, the lasso, ridge regression, forward stepwise selection and PCL. All of those methods can be used with 10 or so lines of R using the packages and examples described in the ISLR book, lab in chapter 6.
In (1) the author use as a cv a pdf that is also a bootloader , and in the comments it seems that he has improved the code. I wonder if he could render dangerzone as futile.

(1) https://news.ycombinator.com/item?id=19344146

Edited: Added in the comments of that post there is a reference to pocorgtfo16.pdf: is valid as a PDF document, a ZIP archive, and a Bash script that runs a Python webserver which hosts Kaitai Struct’s WebIDE which, allows you to view the file’s own annotated bytes. The zip archive has further resources to insane reversing deep dives, code to study and more.

[2] https://www.alchemistowl.org/pocorgtfo/:w

Re:New Features in Java 14 on Wednesday March 04, 2020 @04:58AM (#22482472)
From your blog (5.5 years using scala) it seems that null is a problem in scala:

Scala avoids the biggest pitfall of multiple inheritance elegantly, by only allowing one parent to have a constructor; classes may only be the first parent, and traits can't have constructors that take arguments. Unfortunately this doesn't mean they don't need to be initialized; in particular, vals in a mixed-in trait will be null if you try to access them in an earlier constructor.[2] As with any null, in the worst cases you may not get an error until much later on.

I have read the paper and there are plenty of useful references and points: related work, the 11 CNN based image generators models, and the discussion part.

But sadly I could not obtain a clear picture of what is the difference between their detector and a baseline one. There are some minor points and references about upsampling, downsampling, resizing, cropping and fourier spectra comparison across generators, but those seems to be just comments and comparison and not crucial points in the construction of the detector. Furthermore data augmentation doesn't play a big role, they say that it usually improves (a little) the detector.

As a math person I like to get some more meat from papers, but here it seems that little tricks allow then to win the game. Perhaps that is the way (little or no math involved) to make advances. Well, at least they say that shallow methods modify the fingerprint of the fourier spectra so that now you can't detect which is the generator of the image.

Perhaps the "universal word" was what captured my attention.

Older »