News for hackers. Stuff that matters.
Re:A.I. Companies Agree to Safeguards After Pressure from th... by EGreg on Sunday July 23, 2023 @01:07PM (#36837319)
Alright let’s see

As a Large Language Model, I can run do in 2 seconds what takes you 2 hours… bitch! (Is something I would never say.)

Also, as of my knowledge cutoff date in September 2021, since there is no official public API for Hacker News, I can demonstrate how you can use the unofficial "Hacker News API" by Firebase to fetch comments and analyze them using Perl. Please note that this example uses a third-party API, and the availability or functionality of this API might change over time. Always review and comply with the API's terms of use and documentation before using it in your application.

To perform HTTP requests in Perl, we'll use the LWP::UserAgent module, and for parsing JSON responses, we'll use the JSON module. If you don't have these modules installed, you can install them using CPAN.

Below is an example Perl script that fetches the top 10 comments from the Hacker News API and analyzes them by calculating the average comment length, identifying the most common words, and highlighting comments containing specific keywords:

  perl
#!/usr/bin/perl

  use strict;
  use warnings;
  use LWP::UserAgent;
  use JSON;

  # Function to fetch JSON data from a URL
  sub fetch_json {
    my ($url) = @_;
    my $ua = LWP::UserAgent->new;
    my $response = $ua->get($url);

    die "Failed to fetch data: " . $response->status_line unless $response->is_success;

    return $response->decoded_content;
}

  # Fetch top stories from Hacker News API
  my $top_stories_url = 'https://hacker-news.firebaseio.com/v0/topstories.json';
  my $top_stories_data = fetch_json($top_stories_url);
  my @top_stories = decode_json($top_stories_data)->@*;

  # Fetch comments from the top 10 stories
  my $comments_limit = 10;
  my @comments;

  foreach my $story_id (@top_stories[0..$comments_limit-1]) {
    my $story_url = "https://hacker-news.firebaseio.com/v0/item/$story_id.json";
    my $story_data = fetch_json($story_url);
    
    if ($story_data->{type} eq 'story' && exists $story_data->{kids}) {
        foreach my $comment_id (@{ $story_data->{kids} }) {
            my $comment_url = "https://hacker-news.firebaseio.com/v0/item/$comment_id.json";
            my $comment_data = fetch_json($comment_url);
            push @comments, $comment_data->{text} if exists $comment_data->{text};
        }
    }
  }

  # Calculate average comment length
  my $total_comments = scalar @comments;
  my $total_length = 0;

  foreach my $comment (@comments) {
    $total_length += length($comment);
  }

  my $average_length = $total_length / $total_comments;

  print "Average comment length: $average_length\n";

  # Count occurrences of words in comments
  my %word_count;
  foreach my $comment (@comments) {
    my @words = split(/\W+/, $comment); # Split by non-word characters (e.g., space, punctuation)
    foreach my $word (@words) {
        next unless $word =~ /\w/; # Skip empty strings
        $word_count{lc($word)}++;
    }
  }

  # Sort words by frequency and print the top 5 most common words
  my @sorted_words = sort { $word_count{$b} <=> $word_count{$a} } keys %word_count;
  print "Most common words in comments:\n";
  for my $i (0..4) {
    print "$sorted_words[$i]: $word_count{$sorted_words[$i]   }\n";
  }

  # Define keywords to search for in comments
  my @keywords = qw(perl hacker news code);

  # Find and print comments containing the specified keywords
  print "Comments containing specific keywords:\n";
foreach my $comment (@comments) { if (grep { $comment =~ /\b\Q$_\E\b/i } @keywords) { print "$comment\n\n"; } }

In this script, we first fetch the top stories from the Hacker News API, and then for each story, we fetch the associated comments. The comments are then used to calculate the average comment length, identify the most common words, and display comments containing specific keywords. Remember to review the terms of use and documentation of the API you're using before deploying this script in a real-world scenario.

There are no comments on this comment yet.