News for hackers. Stuff that matters.

BLP4YC

Re:Visualizing technological inventions using text mining on Sunday April 19, 2020 @08:34AM (#22915486)
This post shows how textual information about technological inventions can be used to visualize those inventions (what they can do, how they are related etc.)

It is based on an app that I am working on (see this post: https://news.ycombinator.com/item?id=22847429)

Re:Show HN: NLP-based tool for technology research on Monday April 13, 2020 @11:06AM (#22856458)
Please, tell me. Kidding, but also not. Honestly, I am still unsure. I started working on it because I came across those features in different papers (if I remember correctly - by accident) and thought they were cool. So I started working on it.

But "creators and inventors" goes in the direction I was thinking. Additionally, innovation consultants, tech. advisors, "policy makers" or people who have to decide on technology but lack technical expertise in certain fields, engineers...

Re:Show HN: NLP-based tool for technology research on Monday April 13, 2020 @11:02AM (#22856424)
Yes, absolutely right. The core tech. between crypto-valuations and tech-valuations stayed the same. It was basically: Python + spacy (plus a lot of different supporting libraries).

Some surprising aspects:

For tech-analysis, you can accomplish a lot by using a rule-based parsing because the source texts (e. g. patents) have the same sentence structure. In fact, several researches have shown that patent text follow a certain text structure (e. g. SAO, i. e. Subject-Action-Object).

For crypto, this was for more difficult as the text structures were all over the place.

Also, crypto-analysis ("back then") was very messy because it was difficult to find a trustworthy data set. With technologies you can confine it to Wikipedia, patents, scientific papers. There is still a lot to analyze, but at least you have a somewhat official data set.

Also with crypto you have far less data points per company/token/coin which makes it hard for a machine to not disregard it as noise.

Similarly, with tech-evaluation it seems that - because you get so much data from one document (e. g. one patent) you can often disregard a (big?) portion of it and still end up with good results.

Additionally, it seems to me that crypto-analysis was supposed to be far more numbers-heavy (how much funding etc.) and thus the tolerance for error was relatively small. E. g. if you miss one funding (out of three) you can change the company's valuation up to 50%. This happened to me basically all the time which was super frustrating.

The last surprising fact was how difficult and complicated keyword extraction is. For crypto evaluation I just went with relative word frequency (the more a word appears in a text the more important it becomes, assuming it does not appear in all the documents). However, as I have learnt with tech-evaluation, there are maybe four of five strategies for keyword extraction. And this is still an area where I have not found a solid solution for my NLP-case.

Finally, after all the reading that went into building researchly (which is relatively little) I have realized that I know significantly less about NLP than I have initially thought. It still fascinates me what kind strategies/algorithms people come up with.

Re:Show HN: NLP-based tool for technology research on Monday April 13, 2020 @10:44AM (#22856269)
Thanks! I have been experimenting with NLP for two/three years now and this finally something somewhat useful. So your comment makes me really happay!

Thanks for your feedback regarding the main page. I also like the design, but not because I made, but because the developers behind the templates did such a great job: https://startbootstrap.com/themes/sb-admin-2/ + https://blackrockdigital.github.io/startbootstrap-landing-pa...

I probably should give them credit more prominently (right now, I do it only in the source code).

Regarding the "try now": I have never thought that this will be an issue. On the contrary, I thought that different data across different features will enable people to better understand the variety of the product. But what you are saying makes absolute sense. I have added your points to my todo.

Again, thanks a lot for your feedback - it really helped.

Re:Show HN: NLP-based tool for technology research on Monday April 13, 2020 @01:30AM (#22853424)
It is a combination of:

How interesting I find the paper's ideas.

Would what the paper proposes work with large data sets and can what the paper proposes be (fully) automated.

Can I implement it (do I understand it enough, do I have access to the data)?

Re:Show HN: NLP-based tool for technology research on Sunday April 12, 2020 @10:28AM (#22848458)
To avoid any confusion: I did not come up with most of these features. They are based on these papers (hopefully I have not forgotten any):

* Exploring technological opportunities by linking technology and products: Application of morphology analysis and text mining (Byungun Yoon a,⁎, Inchae Park a, Byoung-youl Coh b) * Technology opportunity discovery (TOD) from existing technologies and products: A function-based TOD framework (Janghyeok Yoon a, Hyunseok Park b, Wonchul Seo c, Jae-Min Lee d, Byoung-youl Coh d,⁎, Jonghwa Kim a,⁎⁎) * Investigating technology opportunities: the use of SAOx analysis (Kyuwoong Kim1 · Kyeongmin Park1 · Sungjoo Lee1) * Identification and monitoring of possible disruptive technologies by patent-development paths and topic modeling (Abdolreza Momeni a, Katja Rost b,⁎) * A New Product Growth for Model Consumer Durables (Frank M. Bass) * Identifying rapidly evolving technological trends for R&D planning using SAO-based semantic patent networks (Janghyeok Yoon • Kwangsoo Kim) * Innovation hotspots in food waste treatment, biogas, and anaerobic digestion technology: A natural language processing approach (Djavan De Clercq a, Zongguo Wen a,⁎, Qingbin Song b) * TrendPerceptor: A property–function based technology intelligence system for identifying technology trends from patents (Janghyeok Yoon, Kwangsoo Kim)

Re:Show HN: NLP-based tool for technology research on Sunday April 12, 2020 @07:16AM (#22847449)
What it does: Uses NLP to extract technology-related information form patents, Wikipedia etc. and then analyzes these technologies.

Technology-related information are: functions (what the technology can do) and properties/components (of what is this technology made up)

Ask HN: Business Intelligence tools to integrate into website

Hi,I want to develop a web-based 'business intelligence' app to analyze and visualize data. CB Insights (https://cbinsights.com) comes very close to it. Some features are: visualize company information, add companies to a watch-list, send alerts etc.For starters, I am looking for an efficient way to visualize time series, but I cannot find anything that seems to work for large amounts of data...

( Read More... | 1 comment | 4 points | on HN )
Care to ELI5?

Older »