HackerNews

Show HN: AI product evaluation methodoloy at huby

There are a number of AI benchmarks but I feel they aren't practical when it comes to evaluating AI products. Several months back we started working on designing a methodology to independently evaluate AI products. After conducting a good bit of research, we tested the methodology on a few different categories of products and improvised it. If this is an area of interest to you, may I request you to review this methodology and help us with feedback. A quick blurb on the structure of the methodo...
HackerNews

KiroCrew – Amazon's AI Harness

Article URL: https://kiro.dev/crew/ Comments URL: https://news.ycombinator.com/item?id=49176063 Points: 2 # Comments: 2
Reddit

Reactive Play: Achieved!! Experimenting with Atari Breakout [R]

Six months ago I started experimenting with PPO and Breakout as a way to learn about Machine Learning and Reinforcement Learning. After a few experiuments just trying to get high scores, it bothered me that everything was a "memorized" script rather than reactive play, like a human would play. Thus began my journey to try and convince PPO to actually track the ball instead of focusing on scoring points. I read a lot of articles and tried a lot of things. After 124 PPO experiments on At...
Reddit

Do ACs also give scores? [D]

This is my first time submitting to NeurIPS. Are ACs also supposed to give ratings during the Phase 2 (author-reviewer discussion session)? I have received the meta-review, but have not received any comments from the AC yet, and was wondering whether this is the standard!
Reddit

NeurIPS 2026 post-rebuttal score distribution poll [D]

As the title suggests, because there's no data on Papercopilot yet, and people have been talking about the scores being lower in general than last year, I thought it could be interesting to survey the average score distribution after the rebuttal phase (not considering confidence weights). Very rough and simple poll (I also realize there's a self-selection bias in there). Cast your vote here: https://loppy.be/poll/yczuv8yo Thanks! Edit: the trolls have taken over, never mind any notion...
I created an autonomous boxing benchmark [D]
Reddit

I created an autonomous boxing benchmark [D]

I created an AI boxing match to test the decision speed, adaptability and strategy. I fed the LLMs with data about the current match and if they have vision, they will get even more data. The match has street rules, anything goes and an AI is not defeated until the ref counts to 10 or they do 50% of their HP in damage after being knocked out. I wanted to create a fun benchmark that isn't just boring problems to be solved. Now I test them while stimulating getting punched in the face. I've b...
Reddit

Missed EMNLP commitment deadline, what can be done? [D]

Asking for a friend: We submitted our paper to ARR May 2026 and got decent scores from the reviewers - 2.5,3,3.5,4. The meta-reviewer gave an overall of 3.5. However, we missed the deadline to commit our work to EMNLP! On our Saturday (we live in the eastern half of the globe), we saw the EMNLP 2026 page on open review with deadline set as Aug3, 11:59PM UTC-0. Apparently, a mail had been sent by ARR on our Sunday regarding committing our work to EMNLP but we didn’t check our mail on the holi...
Reddit

A question on ICLR and NeurIPS deadlines, and OpenReview [D]

After a very silent discussion period, we are in a very confused state with regards to NeurIPS, and really unsure what to make of everything. We do not wish to withdraw the submission since we have no idea what the reviewers and AC think of the paper, having deserted the conversation after a hopeful set of initial reviews. As of currently, ICLR abstract submission deadline is before the NeurIPS results announcement. Are we allowed to resubmit as an ICLR abstract, or will OpenReview flag this and...
Reddit

The Downsides of LLM-Generated Peer Reviews [D]

Having used LLMs to assist with reviews, and also having received reviews that appear to rely heavily on LLM-generated text, I have noticed two recurring problems. 1. The endless search for uncontrolled variables LLMs are very good at identifying additional variables that were not explicitly controlled. The problem is that many of these variables have little realistic chance of changing the paper’s main conclusion. For any experiment, it is possible to generate an almost unlimited list o...
All Bait and No Fish
🌍 HackerNoon

All Bait and No Fish

Job-seeking in tech isn't the same as it used to be - and I'm not talking about the glory days of 2020. Today I'm sharing a few thoughts about how to adapt.