textmode

correction:

sed -n '/pattern/=' file|yy092|sed -nf/dev/stdin file

Correction:

      /* remove HTTP headers from multiple gzip or single zip from stdin */
    
     int fileno (FILE *);
     int setenv (const char *, const char *, int);
     #define jmp (yy_start) = 1 + 2 *
     int x;
    %option nounput noinput noyywrap
    %%
    HTTP\/[\40-\176]+\x0d\x0a x++;
    [\40-\176]+:[\40-\176]+\r\n if(!x)fwrite(yytext,1,yyleng,yyout);
    \x0D\x0A if(!x)fwrite(yytext,1,yyleng,yyout);x=0;
    %%
    int main()
    { 
    yylex();
    exit(0);
    }

Retrieve hostnames, IP addresses and (if available) sitemap URLs from latest Common Crawl.

     ftp -4 https://data.commoncrawl.org/crawl-data/CC-MAIN-2023-50/robotstxt.paths.gz # <-- 180K
     gzip -dc robotstxt.paths.gz \
     |head -5 \
     |sed 's>.*>GET /& HTTP/1.1[]Host: data.commoncrawl.org[]Connection: >;
           $!s/$/keep-alive[]/;$s/$/close[]/' \
     |tr [] '\r\n' \
     |openssl s_client -quiet -connect data.commoncrawl.org:443 \
     |yy054 \
     |zegrep -a '(^Sitemap:)|(^Host:)|(^WARC-Target-URI:)|(^WARC-IP-Address:)' > 1.txt
     exec cat 1.txt

Usage example:

Download NetBSD 1.0 in a single TCP connection.

    y="GET /pub/NetBSD-archive/NetBSD-1.0/source/src10/"
    z="Host: archive.netbsd.org"
    sed '$!s>.*>'"$y"'& HTTP/1.1[]'"$z"'[]Connection: keep-alive[]>;
         $s>.*>'"$y"'& HTTP/1.0[]'"$z"'[]>' << eof \
    |tr '[]' '\r\n' \
    |openssl s_client -quiet -connect 151.101.129.6:443 -servername archive.netbsd.org > http+gzip
    src10.aa
    src10.ab
    src10.ac
    src10.ad
    src10.ae
    src10.af
    src10.ag
    src10.ah
    src10.ai
    src10.aj
    src10.ak
    src10.al
    src10.am
    src10.an
    src10.ao
    src10.ap
    src10.aq
    src10.ar
    src10.as
    src10.at
    src10.au
    src10.av
    src10.aw
    src10.ax
    src10.ay
    src10.az
    src10.ba
    src10.bb
    src10.bc
    src10.bd
    src10.be
    src10.bf
    eof

    yy054 < http+gzip|tar tvzf /dev/stdin

Include an argv[1] will print HTTP headers only

    yy054 print < http+gzip
    yy054 x < http+gzip

Normally I use yy030 but I have been experimenting with this instead.

Seems to be slightly faster and smaller than similar programs from html-xml-utils.

https://www.w3.org/Tools/HTML-XML-utils/man1/

Compile:

   links -no-connect -dump https://news.ycombinator.com/item?id=38727772 \
   |sed '1,4d;77,$d;s/[ ]\{6\}//' \
   |flex -8Cem;cc -O3 -std=c89 -W -Wall -pipe lex.yy.c -static -o yy044
   strip -s yy044

      # NB. not a real cookie
      curl -H "cookie=user=santa&K7RGzmUtAoKv9OIRMfQ9bfwYpiDEuypp" -siA "" \
      https://news.ycombinator.com \
      |host=news.ycombinator.com/ yy044 r \
      |sed -n 's/&/\&/g;/vote/p'

   /* chunked transfer decoding */
   
    #define echo do{if(fwrite(yytext,(size_t)yyleng,1,yyout)){}}while(0)
    #define jmp (yy_start) = 1 + 2 *
    int fileno (FILE *);
    int ischunked,chunksize,count;
   xa "\15"|"\12"
   xb "\15\12" 
   xc "HTTP/0.9"|"HTTP/1.0"|"HTTP/1.1"
   xd [Cc][Hh][Uu][Nn][Kk][Ee][Dd]
   xe [0-9a-fA-F]+\r\n
   xf [0-9a-fA-F]*\r\n
   %option noyywrap nounput noinput 
   %s xb xc xd xe xf
   %%
   ^{xc} echo;ischunked=0;jmp xc;
   <xc>^transfer-encoding: echo;jmp xb;
   <xb>\r\n\r\n echo;jmp xe;
   <xb>{xd} echo;ischunked=1;
   <xe>{xf}|{xe} {
   count=0;
   if(ischunked==1)
   {chunksize=strtol(yytext,NULL,16);
   jmp xd;};
   };
   <xd>{xb} jmp xf;
   <xd>. { 
   count++;
   if(count==chunksize)jmp xe;
   echo;
   };
   <xf>^[A-Fa-f0-9]+{xa}
   <xf>{xa}+[A-Fa-f0-9]+{xa}
   <xf>{xb}[A-Fa-f0-9]+{xb}
   %%
   int main(){ yylex();exit(0);}

Below is a short script that downloads and makes a PDF from the image files. No browser required.

The script uses a feature of HTTP/1.1 called pipelining; proponents of HTTP/2 and HTTP/3 want people to believe it has problems because it does not fit their commercialised web business model. As demonstrated by the script below, it has no problems. It's a feature that simply does not suit the online ad industry-funded business model with its gigantic corporate browser, bloated conglomeration web pages and incessant data collection. Here, only 2 TCP connections are used to retrieve 141 images. Most servers are less restrictive and allow more than 100 requests per TCP connection. Pipelining works great. Much more efficient than browsers which open hundreds of connections. IMHO.

    (export Connection=keep-alive
    x1=http://www.minimizedistraction.com/img/vrg_google_doc_final_vrs03-
    x2(){ seq -f "$x1%g.jpg" $1 $2;};
    x3(){ yy025|nc -vvn 173.236.175.199 80;};
    x2   1 100|x3;
    x2 101 200|x3;
    )|exec yy056|exec od -An -tx1 -vw99999|exec tr -d '\40'|exec sed 's/ffd9ffd8/ffd9\
    ffd8/g'|exec sed -n /ffd8/p|exec split -l1;
    for x in x??;do xxd -p -r < $x > $x.jpg;rm $x;done;
    convert x??.jpg 1.pdf 2>/dev/null;rm x??.jpg

    ls -l ./1.pdf

https://news.ycombinator.com/item?id=27490265 <-- yy054

The "gibberish" is GZIP compressed data. "yy054" is a simple filter I wrote to extract a GZIP file from stdin, i.e., discard leading and trailing garabage. As far as I can tell, the compressed file "ee.txt" is not chunked transfer encoded. If it was chunked we would first extract the GZIP, then decompress and finally process the chunks (e.g., filter out the chunk sizes with the filter submitted in the OP).

In this case all we need to do is extract the GZIP file "ee.txt" from stdin, then decompress it:

    printf "GET /ee.txt\r\nHost: stuff-storage.sfo3.digitaloceanspaces.com\r\nConnection: close\r\n\r\n"|openssl s_client -connect 138.68.34.161:443 -quiet|yy054|gzip -dc > 1.htm
    firefox ./1.htm
   

Need to get rid of the leading spaces on all lines except the "int fileno" line. Can also forgo the "here doc" and just save the lines between "flex" and "eof" to a file. Run flex on that file. This will create lex.yy.c. Then compile lex.yy.c.

The compiled program is only useful for filtering chunked transfer encoding on stdin. Most "HTTP clients" like wget or curl already take care of processing chunked transfer encoding. It is when working with something like netcat that chunked tranfser encoding becomes "DIY". This is a simple program that attempts to solve that problem. It could be written by hand without using flex.

The extra "a" is a typo but would have no effect. The "i" is also superfluous but harmless. Without more details on the "gibberish" it is difficult to guess what happened. The space before "int fileno (FILE *);" is required. All the other lines must be left-justified, no leading spaces, except the line with "int main()" which can be indented if desired.

I make most HTTP requests using netcat or similar tcp clients so I write filters that read from stdin. Reading text files with the chunk sizes in hex interspersed is generally easy. Sometimes I do not even bother to remove the chunk sizes. Where it becomes an issue is when it breaks URLs. Here is a simple chunked transfer decoder that reads from stdin and removes the chunk sizes.

   flex -8iCrfa <<eof
    int fileno (FILE *);
   xa "\15"|"\12"
   xb "\15\12" 
   %option noyywrap nounput noinput 
   %%
   ^[A-Fa-f0-9]+{xa}
   {xa}+[A-Fa-f0-9]+{xa}
   {xb}[A-Fa-f0-9]+{xb} 
   %%
   int main(){ yylex();exit(0);}
   eof

   cc -std=c89 -Wall -pipe lex.yy.c -static -o yy045

Yahoo! serves chunked pages

   printf 'GET / HTTP/1.1\r\nHost: us.yahoo.com\r\nConnection: close\r\n\r\n'|openssl s_client -connect us.yahoo.com:443 -ign_eof|./yy045

Corrections:

/int main/{s/input();//;s/return 0/exit(0)/;};/int yywrap/s/return 0/exit(0)/;/%option/s/$/ noinput/

    static void flush(void) {                                                                                                                 
      if (writeall(1, buf, buflen) == -1) _exit(errno);                                                                                     
      buflen = 0;                                                                                                                           
      }                                                                                                                                         
    static void wrch(const char ch) {                                                                                                         
      if (buflen >= sizeof buf) flush();                                                                                                    
      buf[buflen++] = ch;                                                                                                                   
      return;                                                                                                                               
     }                                                                                                                                         
    char inbuf[128];
    int main(int argc, char **argv) {
        long long r, i;
        for (;;) {
            r = read(0, inbuf, sizeof inbuf);
            if (r == -1) _exit(errno);
            if (r == 0) break;
            for (i = 0; i < r; ++i) {
                wrch("0123456789abcdef"[15 & (inbuf[i] >> 4)]);
                wrch("0123456789abcdef"[15 & inbuf[i]]);
            }
        }
        wrch('\n');
        return 0;
    }

    #include <unistd.h>
    #include <errno.h>
    #include <sys/types.h>
    int writeall(int fd,const void *xv,long long xlen)
    {
      const unsigned char *x = xv;
      long long w;
      while (xlen > 0) {
        w = xlen;
        if (w > 1048576) w = 1048576;
        w = write(fd,x,w);
        x += w;
        xlen -= w;
      }
      return 0;
    }
    static int hexdigit(char x)
    {
      if (x >= '0' && x <= '9') return x - '0';
      if (x >= 'a' && x <= 'f') return 10 + (x - 'a');
      if (x >= 'A' && x <= 'F') return 10 + (x - 'A');
      return -1;
    }
    int hexparse(unsigned char *y,long long len,const char *x)
    {
      if (!x) return 0;
      while (len > 0) {
        int digit0;
        int digit1;
        digit0 = hexdigit(x[0]); if (digit0 == -1) return 0;
        digit1 = hexdigit(x[1]); if (digit1 == -1) return 0;
        *y++ = digit1 + 16 * digit0;
        --len;
        x += 2;
      }
      if (x[0]) return 0;
      return 1;
    }

writeall() comes from http://nacl.cace-project.eu and there it is used in a networking program. I removed poll() but only see a very small speed increase so far.

fsync() can be removed too but I'm not seeing any resulting speed increase.

Alas, input is not always a file so mmap() is not an option.

Glad others are starting to articulate this issue. HTTP/3 is derived from HTTP/2. Google's main argument for HTTP/2's existence, its selling point to users, is head-of-line blocking in HTTP/1.1 pipelining. They also complain about the size of repeated HTTP headers.

But no modern browsers actually use HTTP/1.1 pipelining. Interestingly, HTTP/1.1 pipelining works great for non-browser use. Most web servers enable it by default. After all, it works. For example requesting a series of pages from multi-page website, all under a single TCP connection. I have been using HTTP/1.1 pipeplining this way for decades. It is fast and reliable and enables the web to be used as a non-interactive, information retrieval source. It is also 100% ad-free. The user only gets what she requests, nothing more.

As for HTTP headers, privacy-conscious or minimalist users might not send many headers, only the minimum to retrieve the page. That's usually up to three extra lines of text per page for the request headers. (I rarely ever have to send a User-Agent header for HTTP/1.1 pipelining.)

   GET /index.html HTTP/1.1
   Host: example.com
   Connection: keep-alive

Websites (IPs represented by domain names) to which users intentionally connect, i.e., the recognisable names that they type and click on, generally don't serve ads. The ads come from other domains, often other servers. Users generally do not intentionally try to connect to ad or tracking servers. HTTP/[01].x's automatic loading of resources, Javascript and other techniques may be used to make those requests, conveniently under the radar and outside the user's awareness.

Still, under HTTP/1.1, ads, nor Javascript files that trigger requests for ads, generally cannnot be delivered without the user's computer making a request first. Users can and do manage to exercise some control over their computers and they can prevent these non-interactive requests from being sent, from inside and outside the browser.

With HTTP/2 and HTTP/3, the necessity of a user-generated request disappears. As soon as the user "connects" (UDP) to the website's server, the server could for example send a Javascript file to the user's browser which can in turn trigger requests to other domains for ads or the purpose of tracking, all without any preceding request for the ad/tracking-related Javascript file. This is another feature of HTTP/[23] called "server push", but interestingly it is not the feature being used to sell HTTP/[23] to users (i.e., pipelining).

So, how does a user stop unwanted ads being "pushed" upon her in the stream (irrespective of the application, e.g., browser)? I generally don't use a "modern" browser, nor Javascript nor graphics. I like my pipelining outside the browser and free of advertising-related cruft.

It's worth considering that the motivation for speeding up websites via HTTP/[23] is solely for the purpose of speeding the delivery of more ads, more "stealthily", to users. This is a classic case of someone trying to sell you on a "solution" to a problem they themselves have created (or to which they are contributing).

Like an ISP trying to upsell customers to faster internet in order that websites bogged down with ads will "load" faster. When the ISP itself injects ads into pages of websites that are weighed down by ads.

Better 3.sh

   # 3.sh
   #!/bin/sh
   while IFS= read -r x;do sed -n '/\./p;/\./q'|xxd -p |drill -i /dev/stdin 2>/dev/null;done

Correction:

- x=tinydns/root/data;cat 1.zone >> $x/data;cd $x;

+ x=tinydns/root;cat 1.zone >> $x/data;cd $x;

"Instead of fetching a specific site, you fetch blocks of sites."

I have been doing this for many years, putting bulk DNS data in HOSTS and personal use zone files served from loopback addresses. It is easier than ever today with so many sources of bulk DNS data.

DOH now lets users retrieve DNS data from recursive DNS servers (caches) in bulk, using HTTP/1.1 pipelining. Here is a working example: https://news.ycombinator.com/item?id=23242389

Many years ago, I started doing non-recursive (no caches used) bulk DNS data retrieval for speed and also for resiliency in the event of outages. However the privacy gains are obvious. A rough analogy is downloading all of Wikipedia in bulk and browsing articles offline as opposed to making separate requests online for each article and generating all the requisite DNS and TCP/HTTP traffic. Openmoko's Wikireader experimented with the idea of offline Wikipedia.

Not only does the DOH provider get a record of all the user's DNS lookups, she can now associate each request with the particular user program/device that made it.

   Corrections:
   /tcs/s//openssl s_client -ign_eof -connect/;s/.com/&:443/
   s/1|2/titles|urls/
   s/;;1/;;titles/
   s/;;2/;;urls/

One of the websites where one can find this annnoying "infinite scroll" is YouTube channels.

I wrote a quick and dirty script to address this annoyance.

It can be used to output a table of all the video urls and video titles for any YouTube channel.

"yy032" and "yy025" are some utilities I wrote to decode html and transform urls to HTTP for HTTP/1.1 pipelining, respectively.1 Instead of using yy025 and openssl, one could alternatively make a separate TCP connection for each HTTP request, e.g., using something like curl. Personally, I prefer not to make lots of connections when retrieving mutiple pages from the same domain.

Here is a hypothetical example of how to use the script, "1.sh", to make a table of all the video urls and video titles in a channel.

   echo https://www.youtube.com/user/example/videos|sh 1.sh|yy025|openssl s_client -connect www.youtube.com:443 > 2.html
   sh 1.sh urls < 2.html > example.1
   sh 1.sh titles < 2.html > example.2
   rm 2.html
   paste -d '\t' example.1 example.2

   # 1.sh
   case $1 in
   "")
   exec 2>/dev/null
   export Connection=close
   yy025|tcs www.youtube.com |sed 's/%25/%/g'|yy032 > 1.html
   while true;do
   x=$(sed 's/%25/%/g;s/\\//g' 1.html|yy032|grep -o "[^\"]*browse_ajax[^\"\\]*"|sed 's/u0026amp;/\&/g;s/&direct_render=1//;s,^,https://www.youtube.com,');
   echo > 1.html;
   test ${#x} -gt 100||break
   echo "$x" 
   echo "$x"|yy025|openssl s_client -connect www.youtube.com:443 -ign_eof > 1.html
   done;
   rm 1.html;
   ;;-h|-?|-help|--help) echo usage: echo https://www.youtube.com/user/example/videos \|$0 ;echo usage: $0 "[1|2]" \< 2.htm
   ;;1) sed 's/\\//g;s/u0026amp;//g;s/u0026quot;//g;s/u0026#39;//g'|grep -o "ltr\" title=\"[^\"]*"|sed 's/ltr..title=.//'  
   ;;2) sed 's/\\//g;s/u0026amp;//g;s/u0026quot;//g'|grep -o "[^\"]*watch?v=[^\"]*"|sed 's,^,https://www.youtube.com,'|uniq
   esac

You are probably thinking of pipelining in terms of the popular web browsers. Those programs want to do pipelining so they can load up resources (read: today, ads) from a variety of domains in order to present a web page with graphics and advertising.

That never really worked. Thus, we have HTTP/2, authored by an ad sales company. It is very important for an ad sales company that web pages contain not only what the user is requesting but also heaps of automatically followed pointers to third party resources hosted on other domains. That is, pages need to be able to contain advertising. HTTP/1.1 pipelining is of little benefit to the ad ecosystem.

However, sometimes the user is not trying to load up a graphical web page full of third party resources. Here, the HN commenter is just trying to get some HTML, extract some URLs and then download some files. The HTML is all obtained from the same domain. This is text retrieval, nothing more.

If all the resources the user wants are from the same domain, e.g., archive.org, then pipelining works great. I have been using HTTP/1.1 pipelining to do this for several decades and it has always worked flawlessly.

Typically httpd settings for any website would allow at least 100 pipelined requests per connection. As you might imagine, often the httpd settings are just unchanged defaults. Today the limits I see are often much higher, e.g., several hundred.

It is very rare in my experience to find a site that has pipelining disabled. More likely they are disabling Connection: keep-alive and forcing all requests to be Connection: close. I rarely see this.

The HTTP/1.1 specification suggests a max connection limit per browser of two. There is no suggested limit on the number of requests per connection. In terms of efficiency, the more the better. How many connections does a popular we browser make when loading an "average" web page today? It is a lot more than two! In any event, pipelining as I have shown here stays under the two connection limit.

Yeah, I am pretty good with sed. Probably better than you. I have sed versions of all these programs.

I am not using any computers that cannot run flex and cc.

Results usually go to yy025, a program that makes http from urls.

Of course I use grep -o too. This is not a "correct" filter. It is not a perfect regexp for 100% of urls.

However for something as simple and essential (for the author) as filtering urls I do not want to always have to worry about potential differences in shells, different versions of grep or the absence of a grep as I use different computers, different OS or OS versions. I find this more predictable and portable.

Neither customization nor extensibility are goals. For that a scripting language is better suited.

# If are not a Python user or want to try something different (faster), can be done with sh, sed, openssl, curl/wget/etc. plus a simple utility I wrote called "yy025" (https://news.ycombinator.com/item?id=17689152). yy025 is a more generalised "Swiss Army Knife" for making requests to any website. This solution uses a traditional method called "http pipelining".

   export Connection=keep-alive;
   n=1;while true;do
   test $n -le 8||break
   echo https://archive.org/details/computerchronicles?\&sort=-downloads\&page=$n
   n=$((n+1));done \
   |yy025|openssl s_client -connect archive.org:443 -ign_eof \
   |sed '/item-ia\" [^ ]/!d;s,.*=\",,;s/\"//;s,.*,https://archive.org/download/&/&_archive.torrent,' \
   |yy025|openssl s_client -connect archive.org:443 -ign_eof|sed '/Location:/!d;s/Location: //' 
     

   |while read url;do curl -O $url;done

For discussion...

Question: Are these license restrictions on right to disclose benchmarks enforceable?

Question: If they are enforceable, do licensors ever try to enforce them? If not, why?

A little background here: https://danluu.com/anon-benchmark/

For example, this has been posted to HN at least twice recently:

https://clemenswinter.com/2018/07/09/how-to-analyze-billions...

Question: Was the author subject to any restrictions on publication? If yes, did the author seek "permission" from the licensor to publish these findings?

Excerpts from some of the licenses:

2.2. 32 Bit Kdb+ Software Use Restrictions

(c) 32 Bit Kdb+ Software Evaluations. User shall not distribute or otherwise make available to any third party any report regarding the performance of the 32 Bit Kdb+ Software, 32 Bit Kdb+ Software benchmarks or any information from such a report unless User receives the express prior written consent of Kx to disseminate such report or information.

kdb+ on demand - Personal Edition [64-bit]

1.3 Kdb+ On Demand Software Performance. End User shall not distribute or otherwise make available to any third party any report regarding the performance of the Kdb+ On Demand Software, Kdb+ On Demand Software benchmarks or any information from such a report unless End User receives the express, prior written consent of Kx to disseminate such report or information.

This Kdb+ Software Academic Use License Agreement ("Agreement") is made between Kx Systems, Inc. ("Kx") and you, the University, or employee of the University ("End User") with respect to Kx's 64 bit Kdb+ Software and any related documentation that is made available to you in (jointly, the "Kdb+ Software"). You agree to use the Kdb+ Software under the terms and conditions set forth below. This Agreement is effective upon you clicking the "I agree" button below.

1. LISCENSE GRANTS [sic]

1.4 Kdb+ Software Evaluations. End User shall not distribute or otherwise make available to any third party any report regarding the performance of the Kdb+ Software, Kdb+ Software benchmarks or any information from such a report unless End User receives the express, prior written consent of Kx to disseminate such report or information.

Kdb+ software end-user agreement:

By accessing the Kdb+ Software via the Google platform, you are agreeing to be bound by these terms and conditions (which may be updated from time to time) and to the extent you are acting on behalf of a permitted organization that you have authority to act on their behalf and bind them to these terms and conditions.

You may not access the Kdb+ Software if you are a direct competitor of Kx.

4. Benchmark Test Results. User agrees not to disclose benchmark, test or performance information regarding the Kdb+ Software to any third party except as explicitly authorized by Kx in writing.

Current Intel license:

"Unless expressly permitted under the Agreement, You will not, and will not allow any third party to ... (v) publish or provide any Software benchmark or comparison test results."

If the author uses the ad servers to conduct benchmark or comparison tests of Intel software, and the ad networks allow the author to provide or publish the results of those tests, then it could be argued the ad networks are violating their license agreement with Intel. As a preemptive measure to prevent such testing, perhaps the ad networks would block the author's IP address.

The language in the Intel license probably was inspired from similar language first used by Oracle. This language is commonly copied and pasted into many software license agreements.

http://www.eweek.com/c/a/Application-Development/DB-Test-Pio...

Question: Is this type of restriction enforceable?

The only way to answer this is for end-users to challenge it. There was a case where a state attorney general challenged it because it was used in a deceptive way. The AG won. However the AG was not challenging this restriction as an end-user. The Court appeared to suggest the restriction would be unenforceable, but was not asked to decide that question. The question was whether the state's consumers were being mislead. Excerpts of that case below.

   Excerpts from  http://www.leagle.com/decision/2003579195Misc2d384_1519.xml

   195 Misc.2d 384 (2003)

   758 N.Y.S.2d 466

   Supreme Court, New York County.

   January 6, 2003.

     ----------------------------------------------------------------------

   Supreme Court, New York County.

    OPINION OF THE COURT

   MARILYN SHAFER, J.

"Installing this software constitutes acceptance of the terms and conditions of the license agreement in the box. Please read the license agreement before installation. Other rules and regulations of installing the software are: "a. The product can not be rented, loaned, or leased-you are the sole owner of this product. "b. The customer shall not disclose the result of any benchmark test to any third party without Network Associates' prior written approval. "c. The customer will not publish reviews of this product without prior consent from Network Associates, Inc." (Affirmation of Kenneth M. Dreifach, exhibit 2.)

In July 1999, Network World Fusion, an online magazine, published a comparative review of six firewall software products, including Network Associates' Gauntlet. It appears that Network World Fusion sought permission to publish the review of Gauntlet and that Network Associates denied it. Network World Fusion performed the review despite Network Associates' refusal to allow the review of Gauntlet. In response to the unsatisfactory results of the review, Network Associates communicated its protest, quoting the language of the restrictive clause.

This conduct prompted an investigation by the office of the Attorney General of the State of New York.

"This language implies that limitations on the publication of reviews do not reflect the policy of Network Associates, but result from some binding law or other rules and regulations imposed by an entity other than Network Associates."

Assume for the sake of discussion, there is some such entity.

That is, assume some entity (e.g., Oracle, Microsoft, Intel, etc.) has a license restriction prohibiting publication of benchmark results.

Does the Court think that restriction would be enforceable?

"Thus, the Attorney General has made a showing that the language at issue may be deceptive, and as such, the language is not merely unenforceable, but warrants an injunction and the imposition of civil sanctions according to Executive Law S: 63 (12) and General Business Law S: 349."

Is the Court here suggesting that even if the restriction was not deceptive, it is nevertheless unenforceable.

Is it possible to read that sentence as suggesting the restriction has the qualities of being both unenforceable and deceptive.

As to unenforceability, no users challenged the enforceability of the restriction. Until they do, we cannot answer the question of enforceability.

However, as to deceptiveness, this can be a violation of state business laws and give rise to grounds for injunction and civil sanctions. This is what allowed the NY AG to take action on behalf of NY state consumers.

AG won. NA lost.

The Court granted a permanent injunction prohibiting NA from ever including the following notice with its software:

"Installing this software constitutes acceptance of the terms and conditions of the license agreement in the box. Please read the license agreement before installation. Other rules and regulations of installing the software are: "a. The product can not be rented, loaned, or leased-you are the sole owner of this product. "b. The customer shall not disclose the result of any benchmark test to any third party without Network Associates' prior written approval. "c. The customer will not publish reviews of this product without prior consent from Network Associates, Inc.";

The injunction also prohibits NA from "including any language restricting the right to publish the results of testing and review without notifying the Attorney General at least 30 days prior to such inclusion". NA was directed "to provide a sworn certified statement indicating the number of instances in which software was sold on discs or through the Internet containing the above-mentioned language in order for the court to determine what, if any, penalties and costs should be ordered."

"Raspi cannot be used for a semi serious server application due to microsd card write wear issues."

As an end user, I run a personal authoritative DNS server that has small RAM requirements. The RPi (or other SBC) boots to a mfs mounted root, then mounts all directories as tmpfs. Then I remove the SD card.1 As such, the logs for this server, which are automatically rotated and do not exceed 5M in total, are written to RAM.

1 I only use the SD card to boot. The only files on the card are a bootloader, a bootloader config and two kernels, each with an embedded filesystem. If updates are necessary, I make them to one of the kernels at a time. The other is the backup. The bootloader and bootloader config lets me specify which kernel to boot.

http://web.archive.org/web/20031002050746/http://atrey.karli...

What are the industries that have the highest costs for ETL/Data Engineering?

What companies in these industries are interested in reducing their costs for this work?

"costs" as used above includes time expenditures as well as spending money

http://web.archive.org/web/19991023120316/http://www.dbmsmag...

There is an unwritten rule in biology that if you publish a paper that refers to uses of certain reagents that are not commercially available, then you are obligated to provide those reagents to other investigators who read the paper and request them. There can also be an expected obligation that the other researchers will share any data they generate using the reagents with the original authors.

Outside of biology, I have seen many "academic" papers published on computer-related topics that refer to software programs developed by the papers' authors that are crucial to the research but not publicly available. Is there any similar unwritten rule to that in biology where another researcher reading these papers can request a copy of these programs from the authors?

Obviously, in many cases other researchers cannot replicate and verify findings without access to the same research tools used in the published papers.