再保险:关于robots . txt的问题

布莱恩,后>说,库尔特在2015年12月的邮件…在过去的~ > 30 - 60天,我注意到斜方终于改变了这一切。谷歌现在>索引和缓存CVE页面。我们做出了改变,允许索引早在2016年2月,后几个月库尔特指出这个问题。我们道歉都没有回复原来的线程。丹还提到相同的回应你早在今年4月(http://common -漏洞和风险敞口cve - board.1128451.n5.nabble.com/re cvenew -新- cve -罐- 2017 - 04 - 23 - 19 - 00 -数- 1 - td722.html # a727)。>就像你没有问我们关于3 k +保留的惨败让>今天早上我们几个讨论,找出我们>如何处理它。NVD发言时,我们都集体说“地狱耶!”>>The fact that NVD called you out, and has since said they will be > 'ignoring' those IDs, is also very significant in CVE history. This > is the first *real* break that NVD has had from CVE ever. There have > been other breaks the last year+, but they were more pedantic and > favored NVD > over MITRE/CVE, based on the time of entries becoming > public (e.g. NVD published before MITRE did). We are not absolutely certain what concern you have in the case of the RESERVED CVE IDs moving to REJECT status. Please let us know if the following explanation does not clear up your concerns. We have had multiple conversations during Board conference calls regarding the fact that there are many RESERVED CVE IDs within the current CVE list, and there was a general consensus that they should be cleaned up (i.e., REJECT or populate). As you are probably aware, there are multiple reasons that a CVE ID might be stuck in a RESERVED status. One of those reasons could be that the CNA obtained a block of CVE IDs, but never actually assigned some of those IDs to vulnerabilities. As a first step in tackling the larger cleanup effort, we began contacting CNAs in March of this year to determine what CVE IDs they had not used from their previously assigned CVE ID blocks. All but a couple of CNAs responded and pointed out which CVE IDs were not used. In every case, the CVE ID in question moved from a status of RESERVED to a status of REJECT. The CVE IDs in question were moved to REJECT status earlier today. You are correct and Dave at NIST had sent a message in regards to this first step and he was not clear on exactly what the end result would be. Dave and I spoke on the phone, we cleared up the gaps in understanding, and even decided to hold off for a day to give the NIST NVD folks a bit more time to analyze the impact. Dave can correct me if I'm wrong, but we didn't interpret the comment "ignored by the NVD" to mean that the NVD team would not publish the REJECT CVE entries. Our interpretation is that the NVD team does not see a need to analyze the entries and will simply publish them as is, with no significant effort on their part. Regards, Chris Coffin The CVE Team -----Original Message----- From: jericho [mailto: jericho@attrition.org发送:星期四,2017年5月11日,32点:棺材里,克里斯< ccoffin@mitre.org > Cc: Kurt Seifried < kseifried@redhat.com >;cve-editorial-board-list < cve-editorial-board-list@lists.mitre.org >主题:RE:关于机器人的问题。txt重要性:高在星期二,2015年12月8日,棺材,克里斯写道::我们很久以前不允许选择的索引:cve.mitre.org web站点。至少这个决定只是部分:资源约束?CVE蹒跚学步的年时,搜索引擎:索引器非常资源密集型。“决定”是基于垃圾借口,甚至。=)的人跑两个站点的时间跑CVE冠冕,和集中看到登录其中一个(attrition.org,因为1998-10-07),搜索引擎没有资源密集型。减员人员讨论过这个问题,没有阻止任何内容的机器人。txt因为搜索引擎垃圾邮件是礼物,但不沉重。对于那些感兴趣的是互联网历史上…迫使~ /home/admin/util/list.更多美元filter 72.14.203.104 forced.attrition.org images.search.yahoo.com casualgamer.org myspace.com stumbleupon.com f-mai.gif f-bak.gif f-att.gif thefiles.gif panopta.com divinelanguage.com forced ~$ grep -i google /home/admin/util/list.* /home/admin/util/list.bot:googlebot.com /home/admin/util/list.bot:Feedfetcher-Google /home/admin/util/list.filter-old:google.com /home/admin/util/list.filter-old:google.co.jp/search /home/admin/util/list.filter-old:google.de /home/admin/util/list.filter-old:google.fr /home/admin/util/list.filter-old:google.co.uk forced ~$ "list.filter-old" is from 2003-08-25. The limited set of Google domains should be very telling, given the year and traffic generated. We actually *stopped* filtering Google at some point, while ignoring Yahoo early on. Why? Because they were simply not hammering sites and causing any undue burden, to a random desktop machine bought at the local computer store. Those were "ignore displaying those entries in our log parser", not "block them from reaching our web server" via iptables. That was Attrition when it was run on a ~ $500 box bought in 1998 and hosted on a consumer link, compared to MITRE's resources and CVE contract money from the government at the time. So to be clear, MITRE's answer in 2015, is based on people forgetting what it was like in 1997 - 1999. That said, after Kurt's mail in December of 2015... in the last ~ 30 - 60 days, I noticed that MITRE finally changed that. Google is now indexing and caching the CVE pages. Thank you, as a long-time taxpayer funding MITRE's projects, including CVE, to the tune of $1,487,334,000 in MITRE income last year. Good to see you making these small changes to help the industry. : We are currently re-examining this policy and will keep the Board : posted. Except... you didn't. Just like you didn't ask us about the 3k+ RESERVED fiasco that got several of us talking about this morning, figuring out how we'd handle it. When NVD spoke up, we all collectively said "hell yeah!" The fact that NVD called you out, and has since said they will be 'ignoring' those IDs, is also very significant in CVE history. This is the first *real* break that NVD has had from CVE ever. There have been other breaks the last year+, but they were more pedantic and favored NVD over MITRE/CVE, based on the time of entries becoming public (e.g. NVD published before MITRE did). Brian